Prompt Engineer (Excellent English) at graph8

Posted on (6 months ago)

United States
Remote

Industry: Machine Learning

Job Type: Full-time

Experience Level: Mid-Senior Level

Job Description

We're seeking a candidate with a solid grasp of English and Python fundamentals, expertise in LLMOps, familiarity with current prompting trends, and enthusiasm for AI advancements. This role will focus on developing innovative solutions for Voice AI and automation projects, making technology helpful for our clients while ensuring robust, scalable, and monitored LLM implementations.

Responsibilities

  • Develop, test, and continuously optimize prompts tailored to specific client needs
  • Design and implement robust systems for evaluating and monitoring LLM performance in production environments
  • Utilize LLMOps tools to ensure systematic tracking, versioning, and evaluation of prompts and fine-tuning experiments
  • Implement logging and monitoring solutions to track LLM performance, usage patterns, and potential drift
  • Develop and maintain dashboards for real-time monitoring of LLM metrics and KPIs
  • Collaborate with data scientists and ML engineers to integrate LLMs into larger ML pipelines and systems
  • Actively participate in project reviews and creative brainstorming sessions, contributing both technical expertise and innovative ideas
  • Engage directly with clients to understand their requirements, align expectations, and provide technical guidance
  • Stay current with the latest developments in LLM technology, prompting techniques, and LLMOps best practices
  • Contribute to the development of internal tools and frameworks for streamlining LLMOps processes

Requirements

  • Extensive experience with LLM prompting techniques (e.g., RAG, CoT, function calls, few-shot, chaining)
  • Strong proficiency in LLMOps: evaluating and monitoring accuracy, latency, costs, and relevant metrics
  • Familiarity with LLMOps tools for systematic tracking and evaluation of prompts and fine-tuning
  • Expertise in designing and implementing robust evaluation frameworks for LLMs in production
  • Strong proficiency in English
  • Ability to write clean, readable Python code, and capability to understand and modify existing scripts
  • Proficiency with APIs, SQL databases, and Git
  • Solid understanding of data processing concepts, cloud platforms, and machine learning fundamentals
Browse Jobs
Apply