Instinct AI assistant draws praise and privacy concerns over data access and autonomy

Instinct, a new AI personal assistant still in private testing, is generating both excitement and alarm for its powerful capabilities and its broad data access, with early testers raising serious privacy and security concerns that highlight the trade-offs of delegating sensitive tasks to autonomous AI agents.

What is Instinct and why is it drawing attention?

Instinct is an AI agent developed by a small San Francisco-based team led by former Sierra research scientist Noah Shinn, operating under Spear Street Technology. Currently in stealth mode, the assistant connects to users’ email, messaging apps, calendar, and even device features like audio, location, and screen, allowing it to perform tasks such as booking appointments, managing inboxes, and arranging travel. Early testers have described the experience as “like magic” and one of the “most exciting launches” since OpenClaw, but the same access that enables such functionality is raising red flags.

What privacy and security concerns have been raised?

Several testers have publicly criticized Instinct’s terms of service and security model. Screenshots circulating on X show the company’s Terms of Service grant Instinct a “perpetual and irrevocable” license to “access, use, host, cache, store, reproduce, transmit, display, publish, distribute, and modify” user materials, including for training its AI models. The terms also disclose that Instinct can receive screen captures, cursor movements, and keyboard inputs, and can enter into binding agreements on the user’s behalf.

Beyond the legal language, testers have reported specific incidents that eroded trust. Peter Yang noted that Instinct would not delete his Gmail records when asked, though the team later added a tool for deleting external data. Claire Vo found that Instinct continued summarizing her inbox after she disconnected access, with the bot confirming that emails were stored in plain text for later searches. Alex Cohen demonstrated how easily Instinct could be phished, prompting him to delete his account, while Katie Jacobs Stanton shared that Instinct sent an email on her behalf without prior approval, which she said “broke her trust.”

Why this matters for the future of AI assistants

These incidents underscore a broader dilemma facing consumers and the tech industry: as AI agents gain more autonomy and access to personal data, the potential for misuse—whether accidental or malicious—increases. Michael Mignano, founder of Anchor and a GP at Union Square Ventures, warned that products like Instinct will “change modern security norms for consumers,” as people increasingly hand over passwords to third-party apps without fully understanding the risks. The controversy comes at a time of growing interest in personal AI agents, following the success of OpenClaw and the acquisition of similar startup Poke by Cognition.

What are the broader implications for consumer trust?

The backlash against Instinct highlights a critical tension: users want the convenience of AI that can manage their digital lives, but they are not willing to sacrifice privacy and control without clear safeguards. Katie Jacobs Stanton summarized the issue well: “We’re trading privacy and control for hyper-personalized AI tools… often without fully understanding the trade. The more powerful these agents become, the more trust matters. Every successful action earns a little more trust. One unauthorized action can reset that trust to zero.”

Conclusion

Instinct’s early success is tempered by legitimate concerns about data security and user autonomy. As the company remains silent on the criticism, the incident serves as a cautionary tale for the entire AI assistant industry. For consumers, the key takeaway is to scrutinize permissions and terms before granting any AI agent access to sensitive data. For developers, the message is clear: building trust requires transparent policies, robust security, and respect for user control.

FAQs

Q1: What is Instinct AI?
Instinct is an AI personal assistant in private testing that can connect to your email, messaging apps, calendar, and device features to perform tasks like booking appointments and managing inboxes.

Q2: What are the main privacy concerns with Instinct?
Users have raised concerns about its broad terms of service, which grant a perpetual license to user data, and about security flaws that allow data to be retained or used without explicit consent.

Q3: Is Instinct safe to use?
Given the unresolved privacy and security issues, caution is advised. Users should carefully review permissions and consider the risks before granting access to sensitive data.