LLM Training Frameworks and Optimization Engineer

Together AI is building infrastructure to enable efficient and scalable training of large language models (LLMs). They are seeking a LLM Training Frameworks and Optimization Engineer to drive innovations in the development and optimization of distributed training frameworks. The role involves framework development, algorithmic optimization, and scalability/resilience enhancement.

Requirements

• 5+ years of experience in deep learning frameworks, distributed systems, or machine learning infrastructure.
• Expertise in distributed training frameworks (e.g., PyTorch DDP, DeepSpeed, Megatron-LM, TensorFlow XLA).
• Strong understanding of parallelism techniques (e.g., data, tensor, pipeline, and ZeRO-based parallelism).
• Proficient in Python and C++ or CUDA for high-performance computing.
• Experience with memory optimization techniques (e.g., activation checkpointing, gradient sharding).
• Knowledge of training dynamics for large-scale LLMs, including hyperparameter tuning and optimization.

Benefits

• Competitive compensation
• Startup equity
• Health insurance