Last Week in AI - A Weekly Unwind

From 24-March-2024 to 29-March-2024

It was yet another thrilling week in the AI field with advancements that further extend the limits of what can be achieved with AI.

Here are 10 AI breakthroughs that you can’t afford to miss 🧵👇

Stability AI CEO Resigned among other Key Talents 🕴️

Resignations and poaching have shaken up major AI companies. Last week it was Inflection AI and now Stability AI. Emad Mostaque, the CEO and founder of Stability AI, has stepped down from his role for wanting to pursue decentralized AI. Not just him, Stability AI has witnessed the loss of a talent pool in recent months.

Stability AI has appointed interim co-CEOs, Shan Shan Wong and Christian Laforte while they are searching for a permanent CEO.

Sakana AI’s New Foundation Models by Merging Open Models 👨‍👩‍👧

Sakana AI has introduced Evolutionary Model Merge that uses evolutionary techniques to efficiently discover the best ways to combine different opensource models with diverse capabilities for creating new foundation models. Accessing an extensive pool of 500,000+ models from platforms like Hugging Face, their method has identified creative combinations and generated new foundation models.

This effort has produced two state-of-the-art Japanese models and a fast, high-quality diffusion model without the need for specific optimizations, demonstrating the technique’s versatility across various AI domains.

Peeking Inside the Black-box Transformer Models 👀

Researchers at Google DeepMind, OpenAI, and other institutions have developed a novel model-stealing attack that extracts details from “black-box” transformer AI models, such as ChatGPT and Google’s PaLM-2, with remarkable accuracy for as little as $20. By analyzing the models’ last layer through querying their publicly accessible APIs, the team was able to reveal crucial information like the models’ width and total parameter count, challenging the security of these production-level language models.

The technique successfully uncovered the hidden dimensions of OpenAI’s Ada and Babbage models and estimated the size of GPT-3.5 Turbo’s projection matrix, prompting OpenAI and Google to enhance their defenses against such vulnerabilities.

Full Self-Driving for all Tesla Buyers 🚗

Elon Musk has made it mandatory for every new Tesla buyer in North America to experience the company’s Full Self-Driving (FSD) system firsthand before they drive off with their vehicle. This initiative requires employees to install and demonstrate the latest FSD version 12.3.1 to familiarize customers with its capabilities through a short test ride. To get more users onboard, Musk has further announced a one-month free trial of FSD for all U.S. Teslas capable of the system.

Opensource AI wearable device with DIY feature 🙌

Nik Shevchenko has introduced Whomane’s Friend, the first opensource and affordable AI wearable that contrasts with pricier alternatives like Humane’s AI Pin and Ray Ban Smart Glasses. This device, which costs under $100 to assemble, uses just four main components available on Amazon, ensuring ease of assembly and customization.

Whomane’s Friend supports real-time AI audio processing, allows for the addition of custom software and a camera, and is versatile enough for recording, feedback, and software experimentation.

Grok for All Premium Subscribers 🤖

Elon Musk has announced that xAI’s chatbot Grok will be available to all premium subscribers on X later this week. X Premium has three tiers: Basic, Premium and Premium +, with more features available to high-paying subscribers. Currently, Grok is available only to Premium+ subscribers.

Musk is also enticing users to upgrade to Premium by introducing an ‘unhinged fun mode’ to Grok.

New Opensource LLM Beating Mixtral and Llama 2 💪

Databricks has opensourced their latest offering, DBRX, a general purpose LLM leveraging Mixture-of-experts (MoE) architecture, that outperforms all established open source models including Llama-2 70B, Mixtral 8x7B, and Grok-1 on all industry benchmarks. Not just the performance, its speed is impressive with inference up to 2x faster than Llama 2 70B.

It also surpasses GPT-3.5 and is competitive with Gemini 1.0 Pro. Equally good for specific tasks like coding, unlike other models that specialize in one particular area.

xAI announces Grok-1.5 🚀

Two weeks after opensourcing Grok, xAI has released the next iteration, Grok 1.5. It demonstrates a significant improvement in tasks like language understanding, math, and coding tasks from its predecessor. It also introduces an expansive context capability of up to 128K tokens, 16x more than the previous one. Grok 1.5 will soon be available to all X Premium users.

Transformer + Mamba = New AI Marvel 🌟

AI21 Labs has released Jamba, the world’s first production-grade Mamba-based model. It combines the best aspects of Mamba Structured State Space (SSM) models and traditional Transformer technology to tackle the inherent limitations faced by both architectures on their own. With a humongous 256K context window and an architecture designed for efficiency, Jamba not only addresses some of the pressing challenges in AI model development but also opens up new avenues for application and research.

Next Token Prediction for Humanoid Robot Training 🚶‍♂️

Researchers at UC Berkeley have developed a training method for humanoid robots inspired by the next-token prediction in transformer models for language. This approach uses a causal transformer model to predict a robot’s next move based on its previous actions, incorporating diverse data types such as sensory inputs, motor actions, and even filling in gaps from incomplete data.

When tested in San Francisco, a robot trained with this method successfully navigated various surfaces using only 27 hours of walking data, demonstrating its ability to perform tasks not explicitly covered during its training.

Which of the above AI development you are most excited about and why?

Tell us in the comments below ⬇️

That’s all for today 👋

Stay tuned for another week of innovation and discovery as AI continues to evolve at a staggering pace. Don’t miss out on the developments – join us next week for more insights into the AI revolution!

Click on the subscribe button and be part of the future, today!

📣 Spread the Word: Think your friends and colleagues should be in the know? Click the ‘Share’ button and let them join this exciting adventure into the world of AI. Sharing knowledge is the first step towards innovation!

🔗 Stay Connected: Follow us for AI updates, sneak peeks, and more. Your journey into the future of AI starts here!

Shubham Saboo - Twitter | LinkedIn ⎸ Unwind AI - Twitter | LinkedIn