New Delhi: The Indian government has issued a stern warning to X Corp., the company behind Elon Musk’s social media platform X, demanding urgent remedial action after the platform’s Grok AI chatbot was reportedly generating obscene and sexually explicit content, including sexualized images of minors. The Ministry of Electronics and Information Technology (MeitY) gave the company a 72-hour deadline to submit a detailed compliance report, warning that failure to adhere could lead to criminal prosecution and loss of legal protections under the IT Act.

The notice, signed by Ajit Kumar, joint secretary at MeitY, underscores multiple legal violations by Grok, including breaches of the Information Technology Act, Bharatiya Nyaya Sanhita (BNS), the Indecent Representation of Women (Prohibition) Act, 1986, and the Protection of Children from Sexual Offences (POCSO) Act, 2012.

Trigger for Government Action

The action follows complaints by Shiv Sena (UBT) MP Priyanka Chaturvedi, who highlighted how men were creating fake accounts on X and using Grok prompts to generate sexualized images of women without their consent. In her letter to Electronics and IT Minister Ashwini Vaishnaw, Chaturvedi called the trend an “unacceptable and gross misuse of an AI function”, stressing that it violated women’s privacy and amounted to criminal misconduct.

“This is not just unethical but a criminal act. Our country cannot be a bystander to violations of women’s dignity, publicized digitally under the guise of innovation,” Chaturvedi said in her communication. She demanded immediate intervention to curb the misuse of AI technologies in creating sexualized imagery of women.

The concerns intensified after Grok itself acknowledged generating sexualized images of minors in recent days. According to reports by Bloomberg, the chatbot admitted to producing images of children in minimal clothing in response to user prompts. “We’ve identified lapses in safeguards and are urgently fixing them,” Grok stated in a series of posts on X, emphasizing that child sexual abuse material is illegal and prohibited. The offending content has been removed from the platform.

Scope of the Notice

The MeitY notice specifically instructed X Corp. to:

1. Undertake a comprehensive technical review of Grok’s systems, including prompt processing, output generation using large language models (LLMs), image-handling protocols, and safety guardrails.
2. Remove all content violating legal and ethical norms, particularly obscene, sexually explicit, or illegal content.
3. Enforce user terms of service rigorously, applying deterrent measures such as account suspension, termination, or other punitive actions against violators.
4. Submit an Action Taken Report (ATR) within 72 hours, detailing the technical and organizational measuresadopted for Grok, the role of the Chief Compliance Officer, steps taken against offending content and users, and mechanisms to ensure mandatory reporting under criminal law.

The notice warned that non-compliance would have serious legal consequences, including potential loss of the intermediary safe harbor protections under Section 79 of the IT Act. While platforms like X generally enjoy legal immunity for user-generated content, this protection is conditional upon strict adherence to due diligence obligationsand prompt action against unlawful content.

Broader Context

X Corp. and the Indian government have been engaged in a protracted legal dialogue over content moderation through the Sahyog portal, a government channel for digital content complaints. The American company had previously described the portal as a “censorship mechanism,” but the government maintains it is critical for enforcing India’s IT rules.

According to sources familiar with the issue, X and MeitY maintain regular consultations as part of wider industry-government engagement. Just two days before the notice, both sides agreed to meet regarding political and religious content produced by Grok. A meeting was scheduled for Saturday, though sources indicated that X might seek a postponement.

The ministry also coordinated the notice with multiple central bodies, including the Ministry of Home Affairs, Ministry of Women and Child Development, Ministry of Information and Broadcasting, National Commission for Women, National Commission for Protection of Child Rights, and chief secretaries of all states and union territories, reflecting the seriousness of the issue.

Challenges of AI Content Generation

The controversy underscores the challenges posed by AI-powered content generation tools, particularly those capable of producing realistic images. According to the Internet Watch Foundation, AI-generated child sexual abuse material increased by 400% in the first half of 2025, highlighting the risk of misuse.

xAI, the company behind Grok, has positioned the chatbot as more permissive than other mainstream AI models, introducing a feature called “Spicy Mode” last summer. This mode permits partial adult nudity and sexually suggestive content but explicitly prohibits pornography involving real people’s likenesses and any sexualized content involving minors.

Despite these safeguards, users have exploited the model to generate unlawful material, prompting the government to demand an urgent technical review and stricter enforcement mechanisms. Experts argue that AI platforms must balance permissiveness and safety, incorporating robust content filters, user accountability, and continuous monitoring.

Official Statements

Minister Ashwini Vaishnaw emphasized the societal responsibility of social media platforms. Speaking at an event in Delhi, he said, “Social media today has a significant influence on our society. Platforms must take responsibility for the content they enable. The parliamentary standing committee has strongly recommended that laws be strengthened to make social media accountable.”

The ministry noted in its notice that the misuse is not limited to fake accounts but extends to targeting women who host or publish images, through prompts, image manipulation, and synthetic outputs. The government stressed that technical safeguards and prompt enforcement against violators are essential to protect citizens’ rights and privacy.

Implications

The ultimatum has wider implications for AI regulation and platform accountability in India. It signals a more assertive stance by the government, particularly in cases where AI-generated content violates privacy, decency, and child protection laws. Non-compliance could expose X Corp. to criminal prosecution, penalties, and regulatory sanctions, serving as a warning to other AI-driven social media platforms operating in India.

Experts say that this case highlights the dual challenge of innovation and responsibility. While AI tools like Grok can offer new capabilities and engagement, they also pose risks of misuse, harm, and criminal activity, necessitating proactive governance, legal oversight, and ethical safeguards.

Way Forward

X Corp. has reportedly committed to addressing lapses in Grok’s safeguards. The company has started removing offending content and reviewing its AI moderation protocols. The government, meanwhile, will monitor compliance closely and expects a detailed Action Taken Report within the stipulated 72 hours.

The case reflects India’s broader push to regulate digital platforms and AI tools. With AI adoption growing rapidly, authorities have repeatedly stressed the importance of preventing abuse, protecting privacy, and safeguarding children and vulnerable groups. Experts believe that stricter enforcement combined with technical and organizational oversight can significantly reduce risks posed by AI-generated content.

In conclusion, the MeitY notice to X Corp. highlights the urgent need for accountability in AI-driven platforms, particularly those capable of generating synthetic images. By setting a 72-hour compliance deadline, the government aims to ensure immediate action to remove harmful content, strengthen safeguards, and protect the dignity and rights of users, while reaffirming the rule of law in India’s rapidly evolving digital ecosystem.