Sobre

• AI Operations Specialist – LLM & Conversation Systems
• Position Type: Full-Time, Remote
• Working Hours: U.S. Business Hours
• About the Role
• We are looking for a hands-on AI Operations Specialist to own and improve our production AI conversation systems.
• We are an AI-powered SMS lead recovery platform serving telehealth brands, processing millions of messages each month. Our AI systems interact directly with customers, and their performance directly impacts client conversions.
• In this role, you'll onboard new clients, configure AI conversation systems, monitor live conversations, diagnose issues, improve prompts and system behavior, and measure whether your changes actually improve conversion.
• This is not a theoretical AI role. We're looking for someone who has run LLM systems in production and understands how to continuously monitor, troubleshoot, test, and improve them.
• ResponsibilitiesClient Onboarding & AI Setup
• Onboard new clients into the AI conversation platform.
• Set up webhook integrations and client knowledge bases.
• Configure campaigns and conversation workflows.
• Ensure new client systems are properly tested before going live.
• Troubleshoot integration and configuration issues during onboarding.
• Production AI Operations
• Monitor live AI-generated conversations and customer interactions.
• Review conversations to identify errors, inconsistencies, and missed opportunities.
• Diagnose AI failure modes and determine their root causes.
• Identify issues affecting conversation quality and conversion.
• Escalate technical issues when necessary and follow through until resolved.
• Prompt & System Optimization
• Write, edit, and improve prompts that generate customer-facing messages.
• Adjust system instructions and conversation logic based on real-world performance.
• Test changes systematically before and after deployment.
• Maintain documentation of prompt and system changes.
• Continuously improve AI behavior while maintaining accuracy and consistency.
• Performance & Conversion Analysis
• Analyze conversation transcripts and performance data.
• Measure whether system and prompt changes improve conversion.
• Identify patterns across conversations that can inform future improvements.
• Track key performance outcomes and communicate findings to leadership.
• Use data and real customer interactions to guide optimization decisions.
• Founder Collaboration
• Work directly with the founder on daily AI operations and improvements.
• Communicate issues, findings, and recommendations clearly.
• Take ownership of problems from identification through resolution.
• Move quickly while maintaining high standards for customer-facing AI output.
• Required Experience & SkillsMust-Have
• Proven experience running LLM or AI systems in production.
• Hands-on experience monitoring and improving production AI outputs.
• Experience with prompt engineering and system prompt optimization.
• Strong understanding of LLM behavior, failure modes, and iterative testing.
• Experience analyzing AI-generated conversations or customer-facing outputs.
• Ability to work with webhooks, APIs, knowledge bases, or similar integrations.
• Strong analytical and problem-solving skills.
• Excellent written English — required for editing customer-facing AI messages and documenting system changes.
• Strong attention to detail.
• Ability to work independently and take ownership of production systems.
• What Makes You a Strong Fit
• You've actually operated AI systems in production rather than only experimenting with ChatGPT.
• You are comfortable reviewing hundreds or thousands of AI interactions to identify patterns and problems.
• You think in terms of outcomes and conversion, not just whether the AI response sounds good.
• You are methodical about testing changes and measuring their impact.
• You can quickly identify why an AI system is behaving incorrectly and determine how to improve it.
• You communicate clearly and document your work thoroughly.
• You enjoy working directly with a founder in a fast-moving environment.
• What Does a Typical Day Look Like?
• You'll begin by reviewing the performance of live AI conversations and looking for unusual patterns, failures, or opportunities for improvement.
• You may spend part of the day onboarding a new telehealth client, configuring their knowledge base and integrations, and setting up their campaigns.
• You'll also review conversation transcripts, identify issues with AI responses, update prompts or system instructions, test your changes, and monitor the resulting conversion performance.
• Throughout the day, you'll work directly with the founder to discuss findings, prioritize improvements, and make sure production systems continue performing effectively.
• Interview Process
• Initial Recruiter Screening
• Client Interview
• Offer Stage
• #LI-AG1