Skip to content

About

I’m the technical co-founder of BETTORCA, where I’ve served as the primary engineer behind the company’s product. I started as a full-stack developer, but as the product evolved and we learned more from the market, I became increasingly interested in AI/ML research, particularly how to train models to make predictions that create real value for users.

Outside of engineering, I’m usually in the gym, watching sports, looking for vintage pieces, planning a trip, or working on my golf game.

Projects

Active contributor to open-source tools for model training, exploring different approaches through controlled experiments.

LitGPT

Multi-turn fine-tuning that stays focused

Improved multi-turn fine-tuning by focusing training on assistant replies across complete conversations while preserving existing training behavior. Controlled experiments evaluated effects on response quality, context retention, and general capabilities.

torchtitan

More consistent loss across model depth

Added per-depth loss normalization for Mixture-of-Depths training in DeepSeek models to improve consistency during distributed optimization. Analysis covered gradient scaling, padding, and variable-length sequences to identify when normalization helps.

Tools

Languages
PythonCUDAGoRustBashSQL
Frameworks & Libraries
PyTorchXGBoostHugging FaceTransformers, TokenizersFastAPI
Distributed Training
PyTorch DistributedDTensor, DeviceMeshLightning Fabric
Infrastructure
DockerKubernetesONNXMLflowGitHub ActionsRailway
Data
PostgreSQLQdrantRedispandasPyArrow

Contact

marcus.alan.acosta@gmail.com