Fireworks AI breach exposes Notion customer chat transcripts and billing data

A security breach at Fireworks AI has resulted in the exposure of sensitive customer data, including billing information and datasets. The incident subsequently affected downstream customers, with Notion confirming that chat transcripts from its AI service were among the stolen assets.

What happened

Fireworks AI, a platform that provides developers access to open large language models, confirmed that an attacker breached its cloud provider infrastructure. This initial compromise allowed the threat actor to access a Fireworks development system, which in turn granted access to specific customer data. In a security advisory, the company stated that cybercriminals exfiltrated billing information and certain datasets belonging to its clients. Fireworks serves tens of thousands of customers globally, including notable entities such as legal tech firm Harvey and AI coding assistant Cursor.

The ripple effects of this supply-chain attack reached Notion, the popular online document editor and notetaker. Notion disclosed that approximately 1% of its own customer base was impacted by the breach. After Fireworks alerted Notion to the security incident, Notion took immediate defensive measures. On November 3, the company disconnected its systems from Fireworks, deleted any data stored within those systems, and severed all connectivity between the two platforms. Notion later confirmed that the hackers had specifically targeted and stolen chat transcripts from Notion AI customers who utilized the "Insights" feature.

Notion clarified the scope of the exposed data, noting that the transcripts included any input or output written to the chat during model interactions. However, the company also highlighted a significant limitation in the attackers' haul: much of the data was unattributable. It remains unclear which specific customers owned which transcripts, as the stolen logs lacked direct user identification in many cases. This incident marks the second security breach affecting Fireworks in just two months, following a previous incident in October.

Key details

• Attack vector: Hackers breached Fireworks AI’s cloud provider to access a development system.
• Data stolen: Customer billing information, datasets, and Notion AI chat transcripts.
• Impact on Notion: Approximately 1% of Notion’s customers were affected by the data leak.
• Response timeline: Notion disconnected from Fireworks and deleted related data on November 3.
• Attacker motive: Fireworks believes the actors are financially motivated, aiming to sell stolen LLM access tokens.
• Recurrence: This is the second breach at Fireworks AI in two months, with the prior one occurring in October.

Background

This incident highlights the risks inherent in software supply chains, where companies rely on third-party providers for core functionalities. Fireworks AI acts as an intermediary, allowing developers to integrate open-source large language models into their applications without managing the underlying infrastructure themselves. When such a provider is compromised, the security posture of all its downstream customers is potentially weakened, regardless of their own internal security measures. This is known as a supply-chain attack, where vulnerabilities in a vendor’s system create entry points for attackers to reach multiple targets simultaneously.

The specific motivation cited by Fireworks involves the theft of API tokens or access credentials. In the AI industry, compute resources are expensive. Attackers often steal access to these models to resell usage rights to unauthorized third parties. These bad actors can then run intensive AI workloads without paying for the computing power, effectively burning through the victim’s resources. This financial incentive drives many recent attacks against AI infrastructure providers, making them high-value targets for cybercriminal groups.

Why it matters

For teams that run their own software or manage IT infrastructure, this breach underscores the hidden dependencies in modern tech stacks. Even if your organization does not directly use Fireworks AI, you may rely on tools like Notion, Harvey, or Cursor that do. A vulnerability in a sub-provider can expose your data indirectly. This reality challenges the assumption that choosing reputable vendors guarantees data safety. It forces IT leaders to scrutinize not just their direct vendors, but also the vendors those partners rely on, a task that becomes increasingly complex as services interconnect.

Furthermore, the exposure of chat transcripts raises significant privacy and compliance concerns. For businesses using AI features for internal knowledge management or customer support, these logs may contain sensitive proprietary information or personal data. The fact that much of the data was unattributable offers little comfort, as the mere existence of the transcripts in unauthorized hands creates liability. Teams must now consider how long such data is retained by third parties and whether they have the ability to verify its deletion, as Notion did by severing ties and wiping stored data.

What you can do

• Audit third-party dependencies: Map out which external services your critical tools rely on for data processing or AI features.
• Limit data retention: Configure AI tools to minimize the storage of chat histories and disable features that log unnecessary inputs or outputs.
• Review vendor security advisories: Subscribe to security updates from key vendors to react quickly to incidents like disconnection or password resets.
• Enforce strict access controls: Use role-based access to limit who can enable AI features or export data within your organization’s software stack.
• Consider self-hosted alternatives: Evaluate options for hosting sensitive data processing internally to reduce reliance on external cloud providers for critical workflows.
• Monitor for unusual activity: Keep an eye on API usage metrics and billing statements for unexpected spikes that could indicate token theft or misuse.