How AI Risks Can Outpace Safeguards

UPSC Syllabus: Gs Paper 3- Science and Technology

Introduction

Artificial Intelligence (AI) capabilities are advancing rapidly, while safeguards may not always develop at the same pace. Recent cases show users already testing AI for harmful activities in biology, cybersecurity, surveillance and weapons-related work. At the same time, broader research identifies risks ranging from dangerous capabilities and cyberattacks to inequality and concentration of power. The central challenge is to ensure that detection, access controls and governance develop alongside increasingly capable AI systems.

From Hypothetical Risks to Real-World AI Misuse

1. Evidence of actual misuse: Anthropic documented harmful attempts involving cybersecurity, surveillance and biotechnology, moving AI-risk discussions from hypothetical scenarios towards observed misuse.
2. Biological dual-use risk: Biological capabilities can support legitimate research or harmful activities, making it difficult to distinguish beneficial research from potential misuse.
3. Less stringent older-model safeguards: Opus 4 and Sonnet 4.5 had less stringent biological safeguards because Anthropic’s evaluations found them below the capability level for meaningfully assisting sophisticated dangerous biological research.
4. Delayed strengthening of controls: Stronger biological safeguards were introduced with newer models, while Anthropic separately disclosed that blocking biological classifiers were inactive on roughly 133 million contractor exchanges from May 2025 to April 2026.
5. Examples of attempted misuse: Users applied older Claude models to activities including gain-of-function research on chikungunya and a guided-rocket programme, after which Anthropic terminated accounts when potential illicit use became clear.

Limitations of Existing AI Safeguards

1. Classifier limitations: Input and output classifiers cannot always identify harmful intent from individual messages because the same technical knowledge can have legitimate or harmful uses; pins and screws may form a rifle or wheelchair, while a control loop may serve a missile or air-conditioner.
2. Need for contextual patterns: Harmful intent may become clear only after several related interactions reveal a broader pattern, making individual messages difficult to classify reliably.
3. Generation–detection gap: A dangerous output may already be generated before classifiers recognise the wider harmful pattern, creating a gap between generation and intervention.
4. Data exfiltration risk: Users can save harmful outputs offline before their accounts are terminated, so stopping further access cannot recover information already obtained.
5. Limits of monitoring alone: Monitoring individual interactions may not be sufficient for highly dangerous capabilities, raising the need to consider who should receive access in the first place.

The Expanding Spectrum of AI Risks

1. Expert-based risk assessment: A MIT FutureTech–University of Queensland study asked 272 international AI expertsto assess 24 AI risks over the 2025–2030 period.
2. Catastrophic-risk exposure: Under business as usual, experts judged 18 of 24 risk areas to have more than a 10% probability of catastrophic outcomes over the next five years.
3. Scale of catastrophic harm: The study defined catastrophic outcomes as potentially causing more than 1 million deaths, over $100 billion in financial losses, or comparable civilizational-scale intangible damage.
4. Five risks despite mitigation: Under pragmatic mitigation, five areas still had more than a 10% probability of catastrophic outcomes: dangerous capabilities (12%), AI-enabled weapons and cyberattacks (12%), environmental harm (12%), inequality and unemployment (11%), and power centralisation and unfair distribution of AI benefits (11%).
5. Dangerous capability expansion: More capable AI can make difficult activities easier, including persuasion, surveillance, deepfakes, and assistance with chemical or biological weapons.
6. Cyber and weapons exposure: AI can identify software vulnerabilities, generate code and accelerate offensive cyber activities through its strengths in coding, pattern recognition and information synthesis.

The Governance and Responsibility Gap

1. Sectoral vulnerability: Experts identified the information, finance and national security sectors as particularly vulnerable to AI-related risks.
2. Information-sector risks: AI can increase misinformation, disinformation, privacy loss and manipulation, while weakening trust in the information people receive.
3. National-security risks: Increasingly capable AI can create concerns around cyberattacks, weapons development, surveillance and hostile actors using these systems.
4. Financial-sector risks: AI can amplify fraud, cyber risks, market manipulation, privacy breaches and system failures with wider economic effects.
5. Unequal exposure and responsibility: AI users and the general public were judged most vulnerable, while those exposed to risks are often not best positioned to address them.
6. Developer and government responsibility: Experts assigned the highest responsibility to general-purpose AI developers and governance actors, including governments, regulators and standards bodies.
7. Competitive pressure: Companies and countries seeking economic or strategic advantage may have incentives to deploy AI quickly, resist constraints or underinvest in safety, which can intensify other risks.

Way Forward

1. Continuous risk assessment: Organisations should regularly assess what increasingly capable AI systems can do and whether existing governance practices remain adequate.
2. Risk-based prioritisation: Leaders should focus on harms that experts consider both serious and plausible rather than treating every AI risk equally.
3. Business-process review: Organisations should examine where AI creates value, changes wider ecosystems, replaces human tasks or introduces new vulnerabilities.
4. Access-based safeguards: AI governance can learn from export controls by combining restricted access, identified users and end-use conditions with ongoing monitoring of dangerous capabilities.
5. Integrated governance: AI risk should become part of existing discussions on cybersecurity, privacy, safety and business continuity, rather than remaining a separate compliance issue.
6. Continuous adaptation: Safeguards cannot be a one-time adjustment because AI capabilities, methods of misuse and organisational exposure are changing rapidly.

Conclusion

AI misuse is no longer only hypothetical, as documented cases show users already testing advanced systems for harmful purposes. Existing safeguards can detect and disrupt misuse, but they may not prevent every dangerous transfer before detection. At the same time, AI risks extend beyond deliberate misuse into cybersecurity, national security, finance, inequality and power concentration. Effective governance therefore requires safeguards, access controls and oversight to advance alongside AI capabilities.

Question for practice:

Discuss how the rapid advancement of Artificial Intelligence (AI) can outpace existing safeguards and examine the measures needed to manage its emerging risks.

Source: The Hindu ; MIT