Announcing our investment in Etched's Series A! 🎯 Their mission: Solve AI's compute crunch. GPUs are hitting a wall; they’re getting bigger, but not necessarily better. 🗡 Their weapon: Sohu, a transformer-specific chip. 🚀 The potential: 20x faster AI at 1/20th the cost; That means real-time video gen, instant agents, & more. We're excited to be betting big on specialized AI hardware alongside Etched! Read more about their specialized chip (ASIC), Sohu, below.
Meet Sohu, the fastest AI chip of all time. With over 500,000 tokens per second in Llama 70B throughput, Sohu lets you build products impossible on GPUs. Sohu is the world’s first specialized chip (ASIC) for transformers (the “T” in ChatGPT). By burning the transformer architecture into our chip, we can’t run most traditional AI models. But for generative AI models, like ChatGPT (text), SD3 (images), and Sora (video), Sohu has unparalleled performance. One Sohu server runs over 500,000 Llama 70B tokens per second: >20x more than an H100 server (23,000 tokens/sec), and >10x more than a B200 server. We recently raised $120M from Primary Venture Partners and Positive Sum, with participation from Two Sigma Ventures, Skybox Datacenters, Hummingbird Ventures, Oceans, Fundomo, Velvet Sea Ventures, Fontinalis Partners, Galaxy, Earthshot Ventures, Max Ventures and Lightscape Partners. We’re grateful for the support of industry leaders, including Peter Thiel, David Siegel, Thomas Dohmke, Jason Warner, Amjad Masad, Kyle Vogt, Stanley Freeman Druckenmiller, and many more. We’re on track for one of the fastest chip launches in history: - Top hardware engineers and AI researchers have left every major AI chip project to join us. - We’ve partnered directly with TSMC on their 4nm process. We’ve secured HBM and server supply from top vendors and can quickly ramp our first year of production. - Our early customers have reserved tens of millions of dollars of our hardware. As we hit the limits of speed, cost, and scale on GPUs, specialized chips are inevitable. If you want to change the future of AI compute, please join us at www.etched.com/careers. (Benchmarks are from running in FP8 without sparsity at 8x model parallelism with 2048 input/128 output lengths. 8xH100s figures are from TensorRT-LLM 0.10.08 (latest version), and 8xB200 figures are estimated. This is the same benchmark NVIDIA and AMD use.)