Foundations — What AI Safety Is & Why It Matters

artificial general intelligence

Most AI we have known is narrow: a chess program is superb at chess and useless at anything else; a translation system translates but cannot drive a car. Artificial general intelligence, or AGI, names an informal target at the other end of that spectrum, a system with broad, flexible competence that could learn and perform across most of the cognitive tasks people can do, rather than being locked to one domain. The everyday picture is less a smart appliance and more a capable generalist that can pick up new problems the way a person can.

It is important that AGI has no single agreed definition, and you should be suspicious of anyone who states a precise one as settled. Some define it by breadth (can it handle the full range of tasks a human can?), some by an economic bar (could it do most economically valuable work?), some by matching or exceeding human performance across the board. Recent large language models have made the boundary blurrier, since they are strikingly general compared to older systems yet still fail in ways that suggest they are not doing what a human mind does. Whether current trajectories lead to AGI, and when, is among the most disputed questions in the field.

AGI matters for safety because generality and autonomy are exactly what make the alignment and control problems urgent rather than academic. A narrow tool that only translates text cannot pursue goals in the world; a broadly capable, goal-directed system could, which is when worries about instrumental convergence, misalignment, and loss of oversight start to bite. This is why much safety work is framed around the prospect of AGI or near-AGI systems, while remaining honest that timelines, and even whether AGI is the right concept at all, are genuinely uncertain.

A narrow system can beat any human at Go but cannot be asked to plan a birthday party, write a legal brief, and then debug a program; you would need three different systems. The informal idea of AGI is one system that could turn its hand to all of these, the way a capable person can, which is why generality, not raw skill in one game, is the dividing line.

Generality across tasks, not peak skill in one, is what AGI informally points at.

AGI is a fuzzy, contested label, not a measurable threshold a system clearly crosses on a known date. Treat claims that AGI has arrived, or that it is imminent or impossible, as positions in a live debate rather than established facts.

Also called
AGI通用 AI強人工智慧