Unwind AI has moved from Substack to Beehiiv to enhance your reading experience with a better UI and more streamlined content delivery. Our post on July 28, 2024 was the last post on Substack. All previous and subsequent posts are available on our new website with a new URL, which is theunwindai.com.

Unwind AI has also undergone a rebranding to better reflect our mission of providing you with the latest and most relevant AI insights.

We have a fresh logo, which you can see below. This change represents our commitment to AI education, inclusivity, progressiveness, and staying at the forefront of AI advancements.

Thank you for continuously supporting our publication. We commit to keep bringing you valuable AI content through our publication.

Subscribe to Unwind AI

We have a fresh logo, which you can see below. This change represents our commitment to AI education, inclusivity, progressiveness, and staying at the forefront of AI advancements.

Also, we are moving our publication from Substack to the Beehiiv platform. This transition will help us enhance your reading experience with a better UI and more streamlined content delivery.

Today’s edition will be our last post on Substack, and all previous and future posts will be available on our new website with a new URL, which is theunwindai.com.

A small request from you - please take a moment tomorrow to check your spam folder to ensure you continue receiving our updates without interruption.

Thank you for continuously supporting our publication. We commit to keep bringing you valuable AI content on our platform.

Now, let’s dive into our weekly unwind!

It was yet another thrilling week in the AI field with advancements that further extend the limits of what can be achieved with AI.

Here are 10 AI breakthroughs that you can’t afford to miss 🧵👇

OpenAI’s AI Search Rivals Google’s AI Overviews 🌐

OpenAI has announced that it is prototyping “SearchGPT,” a new search feature that combines AI with the web to give you fast and timely answers with clear and relevant sources. SearchGPT will give you synthesized AI-generated answers to your query, based on real-time information from the web. It combines search with conversation abilities, so you can ask follow-up questions to refine your results or go in depth.

It’s been rolled out to a small group of users for feedback, you can apply for the waitlist here.

Google DeepMind’s AI Models Achieve New Feat in Math 🧠

Google DeepMind’s AI models, AlphaProof and AlphaGeometry 2, solved four out of six problems from this year’s International Mathematical Olympiad (IMO), achieving the same level as a silver medalist in the competition for the first time.
AlphaProof excels in algebra and number theory, while AlphaGeometry 2 focuses on geometry. AI progress in math reasoning can aid mathematicians in discovering new insights and solutions.

Google’s Project Oscar for Opensource Developers 🧑‍💻

Google’s new opensource project Oscar simplifies open-source software development by introducing AI agents to ease the workload of maintaining these projects. The project leverages LLMs not to replace coding, but to handle tedious maintenance tasks like processing issues and connecting questions to relevant documentation.

These agents excel at tasks like processing incoming issues, linking them to existing resources, and even identifying duplicate reports. This approach reduces the need for maintainers to constantly monitor and triage new submissions.

Meta Released Llama 3.1 405B with 128k Context 🌟

Meta released its new series of Llama models, Llama 3.1, including the much-anticipated Llama 3.1 405B model. This is the first frontier-level, opensource AI model with state-of-the-art capabilities, competing with the best closed-source models. This new family of models also includes upgraded versions of the 8B and 70B models.

All Llama 3.1 models are multilingual, have a longer context window of 128K tokens, state-of-the-art tool use, and overall stronger reasoning capabilities. Llama 3.1 405B competes strongly with the leading closed model GPT-4o and Claude 3.5 Sonnet, even outperforming them in a few tasks like math and tool use. Llama 3.1 8B and 70B are the best in their class.

Fine-tune Llama 3.1 with NVIDIA’s All-in-one AI Foundry 🧰

NVIDIA has introduced “AI Foundry,” a new platform for enterprises to build and deploy custom generative AI models using the new Meta Llama 3.1 models. This platform offers comprehensive tools for creating domain-specific models, fine-tuning with proprietary or synthetic data, and deploying them using NVIDIA’s optimized infrastructure. Along with this, NVIDIA has also released NeMo Retriever microservices to enhance RAG pipelines by improving response accuracy.

Mistral AI’s New Flagship Model Mistral Large 2 ⛳️

Mistral AI released their new flagship model, Mistral Large 2 with 123B parameters. This model surpasses its predecessor in code generation, mathematics, and reasoning, boasting stronger multilingual support and advanced function calling. Mistral Large 2 is designed for single-node inference with a 128k context window. Notable, it achieves 84% on MMLU, setting a new standard in terms of performance/cost of serving.

Mistral Large 2 is trained on a very large proportion of code. It outperforms much larger models including CLaude 3.5 Sonnet and Llama 3.1 405B, and matches the performance of GPT-4o on HumanEval and MultiPL-E benchmarks.

Cohere’s Rerank 3 Nimble for Faster Enterprise Search 🕵️‍♂️

Cohere has released Rerank 3 Nimble, its newest foundation model in the Cohere Rerank model series, to enhance enterprise search and RAG systems. This model is 3x faster than its predecessor, Rerank 3, while maintaining a high level of accuracy, making it ideal for high-volume applications.

Rerank 3 Nimble reranks very long documents and processes multi-aspect and semi-structured data for user queries. By integrating Rerank 3 Nimble into RAG systems, developers can ensure that only the most relevant documents are passed to the generative language model. This results in more focused content generation and reduced computational costs.

Microsoft’s Unified Database to Boost RAG 🗃️

LLMs rely on fixed training data which limits their ability to access real-time information, leading to inaccurate or outdated responses. RAG addresses this issue by incorporating up-to-date information from external sources but it doesn’t fully solve the problem due to challenges in managing diverse and multimodal knowledge bases.

To address this, Microsoft has introduced a unified database system that can manage and query various data types, including text, images, and videos. This database system seamlessly handles both vector data (representing semantic relationships) and scalar data (traditional structured data), improving information retrieval speed and accuracy for LLMs. The system is opensourced.

Together AI’s Blazing-Fast Inference Engine 2.0 🏇

Together AI is launching its new inference stack, Together Inference Engine 2.0 (V2). This engine offers significantly faster decoding speeds than opensource vLLM and outperforms alternatives like Amazon Bedrock and Azure AI. Further, Together AI is also releasing Together Turbo and Together Lite endpoints, starting with Meta Llama 3, to provide a range of performance, quality, and pricing options.

Hugging Face’s LM That Can Run on Phone 📱

Hugging Face has introduced SmolLM, a new series of small language models (SLMs) designed for performance and efficiency. The SmolLM family comes in three sizes: 135M, 360M, and 1.7B parameters, each excelling in benchmarks against similar-sized models. These models are trained on SmolLM-Corpus, a meticulously crafted dataset also released by Hugging Face. SmolLM is designed to run locally, even on devices like smartphones, making it highly accessible for various applications.

Which of the above AI development you are most excited about and why?
Tell us in the comments below ⬇️

That’s all for today 👋

Stay tuned for another week of innovation and discovery as AI continues to evolve at a staggering pace. Don’t miss out on the developments – join us next week for more insights into the AI revolution!

Click on the subscribe button and be part of the future, today!

Subscribe now

📣 Spread the Word: Think your friends and colleagues should be in the know? Click the ‘Share’ button and let them join this exciting adventure into the world of AI. Sharing knowledge is the first step towards innovation!

Share

🔗 Stay Connected: Follow us for AI updates, sneak peeks, and more. Your journey into the future of AI starts here!

Shubham Saboo - Twitter | LinkedIn ⎸ Unwind AI - Twitter | LinkedIn | Instagram

Awesome LLM Apps | Sponsor Us

1. OpenAI’s AI search feature to search for real-time information on web with AI

2. Google’s opensource project Oscar for opensource developers

3. Google DeepMind’s AI models solve silver-medal standard questions of International Math Olympiad

4. Runway faces backlash, used YouTube videos without permission to train its GEN-3 AI model

5. Chain your AI agents together seamlessly with Flowise v2

& so much more!

Read time: 3 mins

Latest Developments 🌍

OpenAI’s AI Search Rivals Google’s AI Overviews

OpenAI has announced that it is prototyping “SearchGPT”. OpenAI working on an AI-powered search engine has been rumored for the past few months so it doesn’t come as a surprise. SearchGPT is OpenAI’s new search feature that combines AI with the web to give you fast and timely answers with clear and relevant sources. It’s been rolled out to a small group of users for feedback, you can apply for the waitlist here.

Key Highlights:

1. Search Experience - When you type your query, SearchGPT will give you synthesized AI-generated answers based on real-time information from the web. It combines search with conversation abilities, so you can ask follow-up questions to refine your results or go in depth.

2. In-line Citations - Responses have clear, in-line, named attribution and links so you’d know where information is coming from and can quickly engage with even more results in a sidebar with source links.

3. Publishers and Creators - OpenAI aims to highlight high-quality content in a conversational interface, so publishers don’t have to rely merely on search to reach users. Additionally, OpenAI will also let publishers manage how they appear in SearchGPT, giving them more choices.

4. Location for Precise Results - SearchGPT also uses your location to improve the accuracy of your results. You can choose to share more precise location information for better results.

5. Separate from Model Training - It is separate from training their generative AI foundation models. Sites can be surfaced in search results even if they opt out of generative AI training.

6. Opt Out of Training - Just like ChatGPT, OpenAI will use your conversations with SearchGPT to improve the product but you can opt out of it.

OpenAI is not coming just for companies like Perplexity AI, it’s also challenging Google and Microsoft. It can’t be just a coincidence that Google’s stocks plunged 3% the same day OpenAI announced SearchGPT 🤷‍♀️

Oscar AI Agent for Maintaining Opensource Architecture

Google’s new opensource project Oscar simplifies open-source software development by introducing AI agents to ease the workload of maintaining these projects. The project leverages LLMs not to replace coding, but to handle tedious maintenance tasks like processing issues and connecting questions to relevant documentation.

These agents excel at tasks like processing incoming issues, linking them to existing resources, and even identifying duplicate reports. This approach reduces the need for maintainers to constantly monitor and triage new submissions.

Key Highlights:

1. Contextual Search Engine - Oscar uses Google’s Gemini to analyze and understand the context of issues, documentation, and even code changes. This allows it to provide highly relevant information to contributors and maintainers.

2. Natural Language Control - Many open-source projects rely on a collection of tools for managing contributions, tracking bugs, and more. Oscar allows maintainers to interact with these tools using natural language, rather than complex commands or APIs.

3. In-depth Issue Analysis - Oscar doesn’t just surface related information; it tries to understand the issues itself. It can identify issues that require additional information from reporters, tag them based on content, and even utilize tools like git bisect to pinpoint potential bug introductions. This level of analysis helps maintainers prioritize their efforts and address issues more effectively.

4. Open and Adaptable - While currently in development within the Go project, Oscar’s architecture allows other open-source projects to integrate and customize this agent system for their specific needs and workflows, ensuring wider applicability.

Quick Bytes 🤌

1. Google DeepMind’s AI models, AlphaProof and AlphaGeometry 2, solved four out of six problems from this year’s International Mathematical Olympiad (IMO), achieving the same level as a silver medalist in the competition for the first time.
   AlphaProof excels in algebra and number theory, while AlphaGeometry 2 focuses on geometry. AI progress in math reasoning can aid mathematicians in discovering new insights and solutions.
   

2. Updates from OpenAI

   1. Sam Altman has announced that they are rolling out their Voice Mode to plus subscribers next week!

   2. GPT-4o mini is available for developers to fine-tune, starting with Tier 4 and Tier 5 users. Also, the first 2 million training tokens a day are free till Sept 23.

3. The AI video generation company Runway copied data from thousands of YouTube videos, including popular YouTubers like Marques Brownlee, Casey Neistat, and Sam Kolder to train its GEN-3 model, as per a report by 404 Media.
   

4. Google released Gemini 1.5 Flash model, a lighter version of Gemini 1.5 Pro that closely matches its performance. They have expanded Flash’s access to unpaid users also on Gemini web and app, with an increased context window to 32k tokens.
   Google has also released a new feature with Gemini where you can access additional information on topics directly within responses in Gemini. Links to supporting content will be available at the end of a paragraph.

   

Tools of the Trade ⚒️

1. Flowise v2.0: Flowise AI has introduced a new sequential agentic workflow that allows chaining agents, implementing loopback mechanisms, integrating human approval steps, and creating conditional branches. This gives greater control over the flow compared to relying on chat with LLMs.

2. EDA-GPT: An open-source tool for automated data analysis, supporting various data formats like CSV, XLSX, and SQLite. It helps you visualize and analyze both structured and unstructured data, including PDFs and images, through an intuitive interface.

3. ComfyDeploy : Allows product teams to create AI video and image workflows using ComfyUI and deploy them instantly as APIs. It provides a managed platform, team collaboration features, and scalable cloud infrastructure.

4. Awesome LLM Apps: Build awesome LLM apps using RAG to interact with data sources like GitHub, Gmail, PDFs, and YouTube videos through simple text. These apps will let you retrieve information, engage in chat, and extract insights directly from content on these platforms.

Hot Takes 🔥

1. Jagged Intelligence
   The word I came up with to describe the (strange, unintuitive) fact that state of the art LLMs can both perform extremely impressive tasks (e.g. solve complex math problems) while simultaneously struggle with some very dumb problems.
   E.g. example from two days ago - which number is bigger, 9.11 or 9.9? Wrong. ~
   Andrej Karpathy
   

2. So many use-cases would open up if models were more 'opinionated' ~
   Matt Shumer

Meme of the Day 🤡

Everyone is waiting for OpenAI to release something groundbreaking today. Sama will come and give everyone a dose of patience once again. 😂

That’s all for today! See you tomorrow with more such AI-filled content.

Real-time AI Updates 🚨

⚡️ Follow me on Twitter @Saboo_Shubham for lightning-fast AI updates and never miss what’s trending!

PS: I curate this AI newsletter every day for FREE, your support is what keeps me going. If you find value in what you read, share it with your friends by clicking the share button below!

Share Unwind AI

1. Mistral AI’s new flagship model Mistral Large 2

2. Cohere’s new Rerank model for faster enterprise search and RAG

3. OpenAI could lose $5 billion this year and run out of cash in 12 months

4. Stability AI’s new model generates multiple novel-view videos from a single object video

5. Tools for running Llama 3.1 models locally (100% free and without internet)

& so much more!

Read time: 3 mins

Latest Developments 🌍

Mistral’s New Model with Improved Coding & Reasoning 🧠

Mistral AI is on a release spree. They have released their new flagship model, Mistral Large 2 with 123B parameters. This model surpasses its predecessor in code generation, mathematics, and reasoning, boasting stronger multilingual support and advanced function calling. Mistral Large 2 is designed for single-node inference with a 128k context window. Notable, it achieves 84% on MMLU, setting a new standard in terms of performance/cost of serving.

Key Highlights:

1. Coding - Mistral Large 2 is trained on a very large proportion of code. It outperforms much larger models including CLaude 3.5 Sonnet and Llama 3.1 405B, and matches the performance of GPT-4o on HumanEval and MultiPL-E benchmarks.

2. Less Hallucination - To minimize hallucination, the model was fine-tuned to be more cautious and discerning in its responses. It is also trained to acknowledge when it cannot find solutions or does not have sufficient information to provide a confident answer.

3. Instruction Following - Mistral Large 2 shows significant improvement in instruction following, conversational ability, and generating concise and succinct responses, crucial for business applications.

4. Multilingual - Mistral Large 2 excels in multiple languages, including French, German, Spanish, Dutch, Russian, Chinese, Japanese, Hindi, and more. It outperforms Cohere’s Command R+ and Llama 3.1 70B on Multilingual MMLU.

5. Function calling - Mistral Large 2 is equipped with enhanced function calling and retrieval skills. and can proficiently execute both parallel and sequential function calls, making it ideal for complex business applications.

6. Availability - Mistral Large 2 is available to chat and fine-tune on la Plateforme, and via API. Weights for the instruct model are also available on Mistral’s website and HuggingFace.

Cohere’s Rerank 3 Nimble for 3x Faster Enterprise Search 🕵️‍♂️

Cohere has released Rerank 3 Nimble, its newest foundation model in the Cohere Rerank model series, to enhance enterprise search and RAG systems. This model is 3x faster than its predecessor, Rerank 3, while maintaining a high level of accuracy, making it ideal for high-volume applications.

Rerank 3 Nimble reranks very long documents and processes multi-aspect and semi-structured data for user queries. This enhances existing search systems and the efficiency of RAG systems.

Key Highlights:

1. Improved RAG Efficiency - By integrating Rerank 3 Nimble into RAG systems, developers can ensure that only the most relevant documents are passed to the generative language model. This results in more focused content generation and reduced computational costs.

2. Reduced Latency - Rerank 3 Nimble offers a 3x increase in throughput compared to Rerank 3, resulting in significantly faster response times for search queries.

3. Integration with Existing Systems - It can be implemented with just a few lines of code, and easily integrated with existing search systems, whether they utilize traditional algorithms like BM25 or more advanced techniques.

4. Multilingual Support and Data Versatility - The model supports over 100 languages. It also excels at understanding and reranking complex documents, including multi-aspect data like emails and semi-structured data like JSON and tabular formats.

5. Availability - Rerank 3 Nimble is only available on Amazon SageMaker and for on-premise deployments. It is priced the same as existing Rerank 3 models.

Quick Bites 🤌

1. As per a report, OpenAI is projected to lose $5 billion this year and may run out of cash in 12 months unless they raise more funds. Their expenses, including $7 billion for training and inference and $1.5 billion in workforce costs, far exceeds their revenue of $2 billion from ChatGPT.
   

2. X has launched “More About This Account” feature using Grok AI, for paid users, that provides AI-generated summaries about an X handle when a user hovers over the handle or display name.

3. OpenAI-backed AI-powered legal tech startup Harvey has raised $100 million in a Series C round led by Google Ventures, bringing its total funding to $206 million and valuing the company at $1.5 billion. The new funds will be used to enhance AI models, expand services globally, and grow its team.
   

4. Stability AI has released Stable Video 4D which generates dynamic multi-angle videos from a single object video, offering eight different views in about 40 seconds. It’s currently in the research phase and also available on Hugging Face. 

   1. Converts a single object video into multiple novel-view videos from eight different angles.

   2. Generates 5 frames across 8 views in about 40 seconds.

   3. Users can specify camera angles for tailored creative outputs.

Tools of the Trade ⚒️

1. Run Llama 3.1 models locally on your computer (100% free and without internet) with the following tools:

   1. Ollama - An opensource project that provides an easy way to download and run models like Llama 3.1 easily on regular PCs.

   2. LM Studio - A desktop application that lets you run opensource LLMs offline on your computer.

   3. GPT4All - An open-source project that provides tools and software to run powerful LLMs on your computer, without expensive GPU hardware or cloud services.

2. Awesome LLM Apps: Build awesome LLM apps using RAG to interact with data sources like GitHub, Gmail, PDFs, and YouTube videos through simple text. These apps will let you retrieve information, engage in chat, and extract insights directly from content on these platforms.

Hot Takes 🔥

1. A co-worker once said:
   “People shouldn’t learn how to code - they should learn how to solve problems.”
   He was then fired a month later for being horrible at programming. ~
   Jack Forge
   

2. being afraid of existential risk from ai progress is prudent and advisable and if you reflexively started making fun of this viewpoint in the last
   ~two years after ai entered your radar you need to self reflect ~
   roon
   

3. calling it now:
   harvey (the legal AI co) will end up being roadkill on the side of the highway. complete smoke and mirrors company. ~
   emily is in sf

Meme of the Day 🤡

Getting all your apps to finally work with CUDA.

That’s all for today! See you tomorrow with more such AI-filled content.

Real-time AI Updates 🚨

⚡️ Follow me on Twitter @Saboo_Shubham for lightning-fast AI updates and never miss what’s trending!

PS: I curate this AI newsletter every day for FREE, your support is what keeps me going. If you find value in what you read, share it with your friends by clicking the share button below!

Share Unwind AI

1. Meta releases Llama 3.1 405B model and upgraded 8B and 70B models

2. NVIDIA’s new AI Foundry to customize Llama 3.1 models with 2.5x throughput

3. Tesla will begin selling Optimus robot by 2026

4. Recreate your portraits with Meta AI’s “Imagine Me”

5. Create OpenAI-like API for Llama 3.1 and deploy it locally on your computer (100% free and without internet)

& so much more!

Read time: 3 mins

Latest Developments 🌍

Llama 3.1 405B Leads the Opensource AI Charge 🏆

Meta has released its new series of Llama models, Llama 3.1, including the much-anticipated Llama 3.1 405B model. This is the first frontier-level, opensource AI model with state-of-the-art capabilities, competing with the best closed-source models. This new family of models also includes upgraded versions of the 8B and 70B models. All Llama 3.1 models are multilingual, have a longer context window of 128K tokens, state-of-the-art tool use, and overall stronger reasoning capabilities.

Key Highlights:

1. Architecture - Llama 3.1 405B uses a standard decoder-only transformer model architecture with minor adaptations for training stability. Supervised fine-tuning and direct preference optimization were used for high-quality synthetic data generation and performance improvement.

2. Training - Llama 3.1 405B is trained on over 15 trillion tokens, utilizing over 16,000 H100 GPUs.

3. Inference Optimization - The model was quantized from 16-bit (BF16) to 8-bit (FP8) numerics for lower compute requirements and single-server node operation.

4. Performance - Llama 3.1 405B competes strongly with the leading closed model GPT-4o and Claude 3.5 Sonnet, even outperforming them in a few tasks like math and tool use. Llama 3.1 8B and 70B are the best in their class, outperforming Gemma 2, Mixtral models, and even GPT 3.5 Turbo.

5. Building with Llama 3.1 - Llama 3.1 models are available to download on Meta’s website and Hugging Face. Meta has partnered with AWS, Nvidia, Databricks, Groq, Dell, and others to give you the tools for real-time and batch inference, fine-tuning with your own data, evaluating their performance for your specific needs, generating synthetic data, and more.

6. Chat on Meta AI - Llama 3.1 models are also available to chat with on Meta AI. By default, the chatbot uses Llama 3.1 70B. However, you can manually change it to 405B model for complex questions as it is limited to a few responses only.

Fine-tune Llama 3.1 with NVIDIA’s All-in-one AI Foundry 🧰

NVIDIA has introduced “AI Foundry,” a new platform for enterprises to build and deploy custom generative AI models using the new Meta Llama 3.1 models. This platform offers comprehensive tools for creating domain-specific models, fine-tuning with proprietary or synthetic data, and deploying them using NVIDIA’s optimized infrastructure. Along with this, NVIDIA has also released NeMo Retriever microservices to enhance RAG pipelines by improving response accuracy.

Key Highlights:

1. Custom Model Creation - This AI Foundry allows businesses to create custom “supermodels” based on the new Llama 3.1 models with proprietary data or synthetic data. This platform includes tools for synthetic data generation, fine-tuning, evaluation, and deploying guardrails.

2. Data for fine-tuning - Enterprises that need additional training data for creating a domain-specific model can use Llama 3.1-405B and Nemotron-4 340B together to generate synthetic data.

3. Performance Enhancement - AI Foundry integrates NVIDIA NIMs optimized to deliver up to 2.5x higher throughput for Llama 3.1 models compared to standard deployments. This ensures that AI applications can run with lower latency and higher efficiency.

4. Optimized Infrastructure - Powered by the NVIDIA DGX Cloud AI platform, AI Foundry leverages significant compute resources that can scale with changing AI demands. The platform is co-engineered with leading public clouds, providing flexibility and scalability for enterprises to deploy AI models.

5. NeMo Retriever Microservices - NVIDIA has also released NeMo Retriever microservices to enhance response accuracy in RAG. These microservices include models optimized for embedding, QA, and data retrieval for more precise and context-aware outputs.

Quick Bites 🤌

1. Elon Musk, just like always, announced via a comment on X that Tesla plans to start selling its humanoid robot Optimus to other companies by 2026. By next year, Optimus will be in low production for Tesla’s internal use.

2. Five U.S. senators have demanded that OpenAI provide data on its safety and security measures for its AI systems. Following concerns raised by former OpenAI employees like Jan Leike and Leopold Aschenbrenner, they asked CEO Sam Altman about how the company prevents misuse of AI, ensures employee safety, and addresses cybersecurity threats.

3. Meta has expanded Meta’s AI access to more countries and more languages including French, German, and Hindi, across all its platforms including WhatsApp, Facebook and Instagram. New announcements regarding Meta AI chatbot:

   1. “Imagine me” prompts in Meta AI will let you recreate your image based on a photo of you and a text prompt.

   2. You can easily add or remove objects, change them and edit them with just simple prompts.

   3. Meta AI will be rolled out next month on Meta Quest in the US and Canada in experimental mode, replacing the current Voice Commands.

Tools of the Trade ⚒️

1. Llama 3.1 on LM Studio: Create an OpenAI-like API for the newly launched Llama 3.1 and deploy it locally on your computer in just 2 minutes using LM Studio.

2. Glaive: Build custom models using synthetic data. You can train models for any use case by creating a dataset for it, measure performance, and improve the model by updating or transforming the dataset based on feedback.

3. Robbie: AI agent that navigates GUIs to solve tasks for you. It uses OCR, Canny Composite, and Grid to navigate both web and desktop GUIs, making it versatile for various tasks like sending emails and searching for flights.

4. Awesome LLM Apps: Build awesome LLM apps using RAG to interact with data sources like GitHub, Gmail, PDFs, and YouTube videos through simple text. These apps will let you retrieve information, engage in chat, and extract insights directly from content on these platforms.

Hot Takes 🔥

1. Is there any clarity on what X’s Grok is for? This isn’t snark: are spending a lot on training & have talent, but unclear what’s next. If B2B, they haven’t built out their API & not built for commercial users. On consumer, the chatbot is lagging others. Just powering Elonverse? ~
   Ethan Mollick
   

2. Kind of wild to think meta is training and releasing open weights models faster than OpenAI can release closed models. Dude is already cooking llama 4 ~
   anton

Meme of the Day 🤡

That’s all for today! See you tomorrow with more such AI-filled content.

Real-time AI Updates 🚨

⚡️ Follow me on Twitter @Saboo_Shubham for lightning-fast AI updates and never miss what’s trending!

PS: I curate this AI newsletter every day for FREE, your support is what keeps me going. If you find value in what you read, share it with your friends by clicking the share button below!

Share Unwind AI

1. Meta’s Llama 3 405B will close the gap between opensource and closed AI models

2. Patronus AI releases opensource AI hallucination detection model

3. Microsoft’s new study to enhance RAG performance with a unified database

4. Cohere raises $500 million in Series D funding

5. Developer platform to ship reliable LLM agents 10x faster

& so much more!

Read time: 3 mins

Latest Developments 🌍

Llama-3 405B Base Model Outperforms GPT-4o 😳

Llama-3 405B is supposed to be released today but the model got leaked yesterday on Hugging Face. It was later taken down but the leak gave a sneak peek into the model’s impressive features and performance. The base model itself outperforms the state-of-the-art GPT-4o across multiple benchmarks. Here are all the details we could gather from Reddit:

1. Technical Specifications - The Llama-3 model includes 126 hidden layers, 128 attention heads, and a hidden size of 16,384, promising substantial computational power and advanced processing capabilities.

2. Performance Metrics - In benchmark tests, the model outperforms GPT-4o in several tasks, such as MMLU, BoolQ, GSM8K, and HellaSwag.

3. More Features - With a vocabulary size of 128,256 and support for bfloat16 data type, the model is designed to handle long context efficiently, ensuring robust and accurate outputs.

Leading Hallucination Detection Model Beating GPT-4o

While designed to reduce LLM hallucinations, current RAG systems still struggle to produce text grounded in provided information consistently. This unreliability is particularly noticeable in specialized domains like finance and medicine.

Addressing this, Patronus AI has released LYNX open-source hallucination detection models for superior performance and transparency. In benchmark tests, LYNX outperforms existing models, including GPT-4o in detecting hallucinations.

Key Highlights:

1. Performance - LYNX-70B is the largest open-source hallucination detection model to date. It surpasses even GPT-4o in accurately identifying hallucinations. LYNX-70B demonstrated an 8.3% higher accuracy rate than GPT-4o in detecting inaccurate medical information within the PubMedQA dataset.

2. Explainable AI - LYNX doesn’t just provide a score; it offers a rationale behind its evaluation, mirroring human-like analysis. This “reasoning” capability makes the results more transparent and understandable.

3. Tackling Subtle Hallucinations - LYNX excels in a particularly challenging area: identifying subtle and ambiguous hallucinations often missed by other models. This is attributed to its novel training approach that includes techniques like semantic perturbations, where slight meaning alterations are introduced.

4. Introducing HaluBench - To advance research in hallucination detection models in real-world scenarios, Patronus AI has also released HaluBench, a new, publicly available benchmark dataset that encompasses 15,000 samples across diverse domains, including finance and medicine.

Unified Database to Boost RAG and LLM Performance 🗃️

LLMs rely on fixed training data which limits their ability to access real-time information, leading to inaccurate or outdated responses. RAG addresses this issue by incorporating up-to-date information from external sources but it doesn’t fully solve the problem due to challenges in managing diverse and multimodal knowledge bases.

To address this, Microsoft has introduced a unified database system that can manage and query various data types, including text, images, and videos.

Key Highlights:

1. Unified Database System - This novel database system seamlessly handles both vector data (representing semantic relationships) and scalar data (traditional structured data). This allows for complex queries combining both types, 10 to 1,000x faster than traditional methods, improving information retrieval speed and accuracy for LLMs.

2. SPFresh Technology - This vector index utilizes a lightweight protocol to incorporate updates in real-time, eliminating the need for resource-intensive periodic rebuilds and ensuring LLMs are always current.

3. Sparse and Dense Vector Search - This system efficiently handles both sparse and dense vectors, enabling multi-index mixed queries that cater to diverse data types and encoding methods for more comprehensive search and retrieval.

4. Open-source platform - The team has open-sourced their unified database system, MSVBASE, on GitHub, so you can build and with complex RAG systems.

Quick Bites 🤌

1. Toronto-based AI company Cohere has raised $500 million in Series D funding, valuing it at $5.5 billion post-money. The round was led by PSP Investments, alongside a syndicate of new investors at Cisco Systems, Japan’s Fujitsu, and chipmaker Advanced Micro Devices Inc.’s AMD Ventures.

2. Nvidia is reportedly working on a version of its new flagship AI chips for the China market that would be compatible with current U.S. export controls. Nvidia will work with Inspur, one of its major distributor partners in China, on the launch and distribution of the Blackwell chip, named the “B20.”

Tools of the Trade ⚒️

1. Laminar: Build LLM agents using a visual programming interface and integrate dynamic graphs for rapid experimentation. Utilize RAG for semantic search, switch between various LLM providers, and deploy scalable pipelines with a remote debugger and real-time collaboration.

2. everything-ai: A local open-source AI-powered chatbot assistant that supports tasks like RAG, summarization, audio translation, and image generation. It leverages various open-source libraries. 

3. Sparrow: Opensource solution for extracting and processing data from documents and images. It supports local LLM data extraction, providing APIs for transforming unstructured data into structured formats for custom workflows.

4. Awesome LLM Apps: Build awesome LLM apps using RAG for interacting with data sources like GitHub, Gmail, PDFs, and YouTube videos through simple texts. These apps will let you retrieve information, engage in chat, and extract insights directly from content on these platforms.

Hot Takes 🔥

1. The risk of application copilots is that they hide many of the really powerful features of AI to make LLMs feel safe.
   So it is powerful as an initial model for getting companies to adopt AI systems, but it is hard to get real benefits because all the sharp edges are sanded off. ~
   Ethan Mollick
   

2. it's just so weird how the people who should have the most credibility--sam, demis, dario, ilya, everyone behind LLMs, scaling laws, RLHF, etc--also have the most extreme views regarding the imminent eschaton, and that if you adopt their views on the imminent eschaton, most people in the field will think *you're* the crazy one ~
   James Campbell

Meme of the Day 🤡

Not a meme! But this deserved a space

That’s all for today! See you tomorrow with more such AI-filled content.

Real-time AI Updates 🚨

⚡️ Follow me on Twitter @Saboo_Shubham for lightning-fast AI updates and never miss what’s trending!

PS: I curate this AI newsletter every day for FREE, your support is what keeps me going. If you find value in what you read, share it with your friends by clicking the share button below!

Share Unwind AI

1. Together AI announces new inference stack with Turbo and Lite endpoints

2. Run AI on your phone? Hugging Face’s new small language models make it possible

3. Eleven Labs’ latest text-to-speech model for high quality, low latency in 32 languages

4. Opensource AI on-call developer that jumps into incidents & alerts with you

& so much more!

Read time: 3 mins

Latest Developments 🌍

Together AI’s Blazing-Fast Inference Engine 2.0 🏇

Together AI is launching its new inference stack, Together Inference Engine 2.0 (V2). This engine offers significantly faster decoding speeds than opensource vLLM and outperforms alternatives like Amazon Bedrock and Azure AI. Further, Together AI is also releasing Together Turbo and Together Lite endpoints, starting with Meta Llama 3, to provide a range of performance, quality, and pricing options.

Key Highlights:

1. Faster Decoding - V2 boasts a 4x faster decoding throughput compared to open-source vLLM. It also surpasses the performance of commercial solutions like Amazon Bedrock, Azure AI, Fireworks, and Octo AI by a 1.3x to 2.5x.

2. Together Turbo Endpoints - These endpoints prioritize speed and cost-effectiveness while maintaining quality comparable to full-precision FP16 models. They outperform other FP8 solutions and are 17x lower in cost than GPT-4o.

3. Together Lite Endpoints - Designed for maximum affordability and scalability, Together Lite utilizes optimizations like INT4 quantization. These endpoints offer exceptional quality relative to full-precision models, making high-quality AI more accessible.

4. Together Reference Endpoints - Delivering the fastest full-precision FP16 support for Meta Llama 3, these endpoints offer up to 4x faster performance than vLLM. They are ideal for tasks requiring the highest level of quality.

5. Technical Advancements - V2 incorporates cutting-edge research, including FlashAttention-3 kernels, speculative decoding algorithms, and quality-preserving quantization techniques. These ensure optimal performance, accuracy, and efficiency.

Hugging Face Thinks Big, Builds Small (LMs) 🧠

Hugging Face has introduced SmolLM, a new series of small language models (SLMs) designed for performance and efficiency. The SmolLM family comes in three sizes: 135M, 360M, and 1.7B parameters, each excelling in benchmarks against similar-sized models. These models are trained on SmolLM-Corpus, a meticulously crafted dataset also released by Hugging Face. SmolLM is designed to run locally, even on devices like smartphones, making it highly accessible for various applications.

Key Highlights:

1. High Performance - SmolLM consistently outperforms other SLMs in its size category across benchmarks, particularly in common sense reasoning and world knowledge tests. Notably, SmolLM-1.7B surpasses models like Phi1.5 and Qwen2-1.5B.

2. Efficient Training - Despite their strong performance, SmolLM models are trained on less data than some competitors. The 135M and 360M models are trained on 600B tokens, while the 1.7B utilizes 1T tokens, showcasing efficiency in data usage.

3. Focus on Data Quality - SmolLM-Corpus, the dataset used for training, is carefully curated and includes Cosmopedia v2, FineWeb-Edu, and Stack-Edu-Python. This focus on high-quality, education-oriented data contributes to the models’ robust capabilities.

4. Local Deployment - Designed for accessibility, SmolLM models can run locally on devices with limited resources. Hugging Face emphasizes their small memory footprint, making them suitable even for smartphones, and provides ONNX checkpoints for broader compatibility.

Quick Bites 🤌

1. Eleven Labs has introduced Turbo v2.5, their new text-to-speech model supporting 32 languages, including new additions like Vietnamese, Hungarian, and Norwegian. It offers 3x faster performance for many languages and a 25% speed increase for English, enabling high-quality, low-latency conversational AI.

2. OpenAI is in discussions with Broadcom to develop a new AI chip to reduce its dependence on Nvidia and enhance its supply chain. Led by CEO Sam Altman, the aim is to secure the necessary components and infrastructure for running advanced AI models.

3. Google, OpenAI, Microsoft, Amazon, and other major AI companies have formed the Coalition for Secure AI (CoSAI) to promote secure-by-design AI. CoSAI aims to create best practices, address AI security challenges, and provide open-source tools within the OASIS framework.

Tools of the Trade ⚒️

1. Merlinn: AI-powered on-call engineer to provide real-time insights and root cause analysis for production incidents. It integrates with tools like Slack, Datadog, and Jira to manage incidents more efficiently and improve your on-call performance.

2. Guardrails Server: Deploy guardrails for your language models on the cloud, making local deployment to the cloud seamless. It supports the OpenAI SDK and offers cross-language compatibility for use in any language that supports the OpenAI SDK.

3. FlowTestAI: AI-powered opensource IDE designed to create, visualize, and manage API-first workflows. Describe your API workflows in natural language, and it visualizes API interactions, and performs end-to-end contextual testing.

4. Awesome LLM Apps: Build awesome LLM apps using RAG for interacting with data sources like GitHub, Gmail, PDFs, and YouTube videos through simple texts. These apps will let you retrieve information, engage in chat, and extract insights directly from content on these platforms.

Hot Takes 🔥

1. agents will probably generate order of magnitude more revenue than chatbots but both will end up being tiny easter eggs to fund the capex for superintelligence ~
   roon
   

2. I don’t think OpenAI is delaying the new voice mode over moans or meth recipes.
   It’s probably because sufficiently advanced voice AI has the potential to induce psychosis 😵‍💫 ~
   Pliny the Prompter

Meme of the Day 🤡

Founders dropping out of Stanford and CMU PhD programs raising money for their AI startups. With no product, no revenue, and no idea.

That’s all for today! See you tomorrow with more such AI-filled content.

Real-time AI Updates 🚨

⚡️ Follow me on Twitter @Saboo_Shubham for lightning-fast AI updates and never miss what’s trending!

PS: I curate this AI newsletter every day for FREE, your support is what keeps me going. If you find value in what you read, share it with your friends by clicking the share button below!

Share Unwind AI

Here are 10 AI breakthroughs that you can’t afford to miss 🧵👇

OpenAI’s Most Cost-Efficient Small Model 👩‍👦

OpenAI has launched GPT-4o mini, its most cost-effective small multimodal model yet. It is 60% cheaper than GPT-3.5 Turbo and boasts excellent capabilities at a significantly lower price point. It achieves an impressive 82% on MMLU and outperforms GPT-3.5 Turbo in various tasks.

GPT-4o mini supports text and vision in the API, with support for text, image, video, and audio inputs and outputs coming soon. It is now available as API and in ChatGPT for all users in place of GPT 3.5 Turbo.

Tenstorrent Releases New AI Devkits and Workstations 💡

Tenstorrent, the AI chip company led by renowned engineer Jim Keller, has released new devkits and AI workstations built around their latest Wormhole AI accelerators. Their Wormhole architecture is designed to be more powerful and efficient than its predecessor, Grayskull. Manufactured using a 12nm process, it boasts 80 Tensix+ cores each and offers up to 328 TOPS of compute performance.

Tenstorrent is offering two Wormhole-based PCIe cards: the n150 (single chip) and the n300 (dual chip). For those seeking a complete system, the TT-LoudBox and TT-QuietBox workstations each housing four n300 cards are available.

Groq’s Llama-3 Tops Function Calling Benchmarks 🏆

Groq has unveiled two new open-source language models, Llama-3-Groq-70B-Tool-Use and Llama-3-Groq-8B-Tool-Use, specially designed for advanced tool use and function calling. They are released with the same permissive style license as the original Llama 3 models.

Llama-3-Groq-70B-Tool-Use tops the Berkeley Function Calling Leaderboard (BFCL) surpassing all models with a 90.76% overall accuracy. Llama-3-Groq-8B-Tool-Use achieves an 89.06% accuracy, securing the 3rd position on the leaderboard.

AI Search Engine for RAG Apps & AI Agents 🌐

Exa is an AI search engine that finds the best content on the web using embeddings. Exa’s precise content retrieval supercharges RAG pipelines, automates hours of research, and creates high-quality datasets for your niche use-case.

Exa has announced a major upgrade with its 1.5 release. Exa 1.5 features an upgraded index with high-value data types, including scientific research papers, company information, and even tweets. It introduces hybrid search, combining neural search with keyword matching for more targeted results, and an Auto Search with Google fallback.

New Text-to-Video Model Rivaling Sora and GEN-3

London-based company Haiper has released its new video generation model Haiper 1.5 which generates 8-second-long videos from text or image prompts. It can even extend your prior 2 and 4-second videos to 8 seconds, just like Luma Labs Extend feature. Below is a comparison of videos generated by Haiper 1.5, Luma Labs, and Runway GEN-3, for the same text prompt.
”Dragon-toucan walking through the Serengeti.”

Not just this, Haiper also has an integrated upscaler that can upscale videos to 1080p in a single click.

New Specialized Models by Mistral AI 🤖

Mistral AI has released two new specialized models: Codestral Mamba which specializes in code generation, and Mathstral designed for math reasoning and scientific discovery.

Codestral Mamba, a Mamba2 language model, outperforms other 7B models on various coding benchmarks and even competes with larger models including Mistral’s Codestral 22B and Meta’s CodeLlama 34B.

Mistral’s first Mathstral model with 7B parameters is designed for advanced mathematical problems requiring complex, multi-step logical reasoning. It scored 56.6% on MATH and 63.47% on MMLU.

Andrej Karpathy Launches AI + Education Company ✍️

Andrej Karpathy is starting an AI + Education company called Eureka Labs with the goal of creating an AI-native school. Partnering with subject matter experts, the first product will be LLM101n, an undergraduate-level course guiding students to train their own AI.

12x Faster Inference for RAG 📈

Fireworks AI has released FireAttention V2, an optimized CUDA kernel for high-traffic AI applications, significantly improving inference times for long contexts (8K-32K tokens). V2 achieves up to 12x faster inference compared to previous versions, outpacing alternatives like vLLM in both short-medium and long-generation tasks.

FireAttention V2 enhances throughput and latency, crucial for maintaining performance in expanding applications like RAG and multi-agent systems.

Build RAG Apps that can Process 10 Million Words 🧠

Writer, an AI startup, released its AI Studio, enabling enterprises to build AI apps without coding. Recent upgrades include built-in RAG for analyzing up to 10 million words, explainable AI, dedicated modes, voice rewrites, and custom instructions.

These upgrades allow chat apps to upload large files and use graph-based RAG for accurate responses. Explainable AI breaks down complex questions with cited sources, while dedicated modes and customization features enhance user experience and functionality.

Meta is Releasing Llama-3 405B on July 23 🐑

Meta is planning to release the largest Llama-3 model with 405B parameters on July 23. As Meta had announced earlier, Llama-3 405B will be multimodal, multilingual, and have a large context window. Its performance had reached GPT-4 level while it was in training. It’d be the most capable opensource model to date!

Which of the above AI development you are most excited about and why?
Tell us in the comments below ⬇️

That’s all for today 👋

Stay tuned for another week of innovation and discovery as AI continues to evolve at a staggering pace. Don’t miss out on the developments – join us next week for more insights into the AI revolution!

Click on the subscribe button and be part of the future, today!

Subscribe now

📣 Spread the Word: Think your friends and colleagues should be in the know? Click the ‘Share’ button and let them join this exciting adventure into the world of AI. Sharing knowledge is the first step towards innovation!

Share

🔗 Stay Connected: Follow us for AI updates, sneak peeks, and more. Your journey into the future of AI starts here!

Shubham Saboo - Twitter | LinkedIn ⎸ Unwind AI - Twitter | LinkedIn | Instagram

Awesome LLM Apps | Sponsor Us

1. OpenAI releases its smallest and cheapest GPT-4o mini model

2. Jim Keller’s Tenstorrent releases new AI devkits and workstations

3. Mistral AI and Nvidia team up for a powerful new language model

4. Google makes its AI-powered text-to-video presentations tool available

& so much more!

Read time: 3 mins

Latest Developments 🌍

OpenAI’s Most Cost-Efficient Small Model 👩‍👦

OpenAI has launched GPT-4o mini, its most cost-effective small multimodal model yet. It is 60% cheaper than GPT-3.5 Turbo and boasts excellent capabilities at a significantly lower price point. It achieves an impressive 82% on MMLU and outperforms GPT-3.5 Turbo in various tasks. This new model opens doors for developers to build a wider range of AI applications without breaking the bank.

Key Highlights:

1. Multimodality - GPT-4o mini supports text and vision in the API, with support for text, image, video, and audio inputs and outputs coming in the future.

2. Context Window and Output Tokens - The model has a context window of 128K tokens, supports up to 16K output tokens per request, and has knowledge up to October 2023.

3. Slashed Prices - Priced at $0.15 per million input tokens and $0.60 per million output tokens, GPT-4o mini is considerably cheaper than previous models.

4. Performance - GPT-4o mini excels in academic benchmarks for textual intelligence and multimodal reasoning. It outperforms GPT-3.5 Turbo as well as other models like Gemini Flash and Claude Haiku across MMLU, math, coding, and multimodal reasoning tasks.

5. Availability - GPT-4o mini is now available as a text and vision model in the Assistants API, Chat Completions API, and Batch API. In ChatGPT, Free, Plus and Team GPT-4o will replace GPT-3.5. Enterprise users will also have access starting next week.

Tenstorrent Releases New AI Devkits and Workstations 💡

Tenstorrent, the AI chip company led by renowned engineer Jim Keller, has released new devkits and AI workstations built around their latest Wormhole AI accelerators. This is the first time they’re offering complete workstations alongside their devkits. Tenstorrent’s Wormhole architecture is designed to be more powerful and efficient than its predecessor, Grayskull. Their commitment to a RISC-V processor ISA and open-source software sets them apart in the AI acceleration market.

Key Highlights:

1. Powerful Hardware - The Wormhole accelerators, manufactured using a 12nm process, boast 80 Tensix+ cores each and offer up to 328 TOPS of compute performance.

2. Devkits and Workstations - Tenstorrent is offering two Wormhole-based PCIe cards: the n150 (single chip) and the n300 (dual chip). For those seeking a complete system, the TT-LoudBox and TT-QuietBox workstations each house 4 n300 cards, providing a total of 8 Wormhole chips in a powerful 2x4 mesh configuration.

3. Workstations - The difference between both workstations lies in the cooling and CPU configuration, with the TT-LoudBox opting for air cooling and Intel Xeon processors, and the TT-QuietBox utilizes liquid cooling and an AMD EPYC processor.

4. Availability - The n150, n300, and TT-LoudBox are currently available for purchase on Tenstorrent’s website. The TT-QuietBox is available for pre-order and is expected to ship within the next 8-10 weeks.

Mistral AI’s New Model Surpasses Gemma 2 and Llama 3 💪

Mistral AI has released Mistral NeMo, a new 12B language model developed in partnership with Nvidia. Available under the Apache 2.0 license, this powerful model boasts a 128k context window and excels in reasoning, knowledge, and coding tasks for its size. Mistral NeMo is designed to be easily integrated into existing systems as a replacement for the Mistral 7B model.

Key Highlights:

1. Performance - Mistral NeMo outperforms Gemma 2 9B and Llama 3 8B on reasoning, commonsense, and truthfulness benchmarks. It also exhibits remarkable fluency in numerous languages, including French, German, Spanish, Italian, Portuguese, Chinese, Hindi, and more.

2. Tokenization - The model utilizes “Tekken,” a new tokenizer trained on over 100 languages. Tekken compresses text and code more efficiently than other tokenizers, particularly in languages like Chinese, Korean, and Arabic.

3. Instruction Following - The instruction-tuned version demonstrates major improvements in following complex instructions, logical reasoning, multi-turn conversations, and generating high-quality code compared to Mistral 7B.

4. Availability - Weights for both the base and the instruct models are hosted on HuggingFace. The model is also packaged in a container as Nvidia NIM inference microservice.

Quick Bites 🤌

1. Meta is withholding its next multimodal AI model from the EU due to “lack of clarity from regulators there.” This follows a similar move from Apple as they too won’t be releasing Apple Intelligence in the EU due to similar reasons. (Source)
   

2. Knowledge graphs for better RAG capabilities are catching significant momentum. Samsung is acquiring UK-based knowledge graph startup Oxford Semantic Technologies to enhance its AI capabilities. By utilizing knowledge graphs to semantically map users’ information, Samsung aims to provide personalized and secure on-device AI experiences. (Source)
   

3. TSMC’s more than 50% of revenue now comes from high-performance computing, driven by strong demand for AI chips. The company reported better-than-expected profit yesterday and raised its revenue growth outlook for 2024, expecting AI spending to remain high. (Source)
   

4. Google has started testing Gemini-powered video presentations Vids app in Google Workspace Labs. You can upload your documents, slides, voiceovers and video recordings, and Gemini with create video presentations with stock footage. This is not to be confused with text-to-video models like Google’s Veo. (Source)

Tools of the Trade ⚒️

1. TalkTastic: Using your voice instead of manually typing everything in any app on macOS. It offers accurate dictation and smartly rewrites even when you stutter, based on the context of what you see on your screen. It ensures privacy by only listening and taking snapshots when you command.

2. Traceloop: Monitor your LLMs for issues like hallucinations and malfunctions, for consistent and reliable outputs. It provides real-time alerts, backtesting capabilities, and debugging tools for prompts and agents, all with easy installation and seamless integration.

3. LangSmith Playground: Quickly build and compare multiple prompts and model configurations side-by-side without switching tabs. It lets you test, iterate, and evaluate changes efficiently in a single view.

4. Awesome LLM Apps: Build awesome LLM apps using RAG for interacting with data sources like GitHub, Gmail, PDFs, and YouTube videos through simple texts. These apps will let you retrieve information, engage in chat, and extract insights directly from content on these platforms.

Hot Takes 🔥

1. We don’t think anymore, we just hack LLMs. ~
   Pedro Domingos
   

2. Years of SaaS software have messed with people’s heads when thinking about AI. Companies & schools are used to outsourcing R&D - software vendors build products & organizations implement them.
   For AI, no one has special insight. Folks will need to do R&D themselves to get ahead ~
   Ethan Mollick

Meme of the Day 🤡

POV: When you were waiting for GPT-5 but GPT-4o-mini is coming.

That’s all for today! See you tomorrow with more such AI-filled content.

Real-time AI Updates 🚨

⚡️ Follow me on Twitter @Saboo_Shubham for lightning-fast AI updates and never miss what’s trending!

PS: I curate this AI newsletter every day for FREE, your support is what keeps me going. If you find value in what you read, share it with your friends by clicking the share button below!

Share Unwind AI

1. Groq’s Llama-3 model for advanced tool use and function calling

2. AI search engine built specially for RAG apps and AI agents

3. Cohere toolkit now builds AI apps with Interactive HTML and Multi-step Tool-use

4. New text-to-video model that generates 8 seconds-long videos in 1080p

5. Python and React AI assistant powered by Claude 3.5 Sonnet

& so much more!

Read time: 3 mins

Latest Developments 🌍

Groq’s Llama-3 Tops Function Calling Benchmarks 🏆

Groq has unveiled two new open-source language models, Llama-3-Groq-70B-Tool-Use and Llama-3-Groq-8B-Tool-Use, specially designed for advanced tool use and function calling. These models are available on GroqCloud Developer Hub and Hugging Face. They are released with the same permissive style license as the original Llama 3 models.

Key Highlights:

1. Unprecedented Performance - Llama-3-Groq-70B-Tool-Use tops the Berkeley Function Calling Leaderboard (BFCL) surpassing all models with a 90.76% overall accuracy. Llama-3-Groq-8B-Tool-Use achieves an 89.06% accuracy, securing the 3rd position on the leaderboard.

2. LLM Routing Strategy: Groq suggests a hybrid approach where developers can implement an LLM routing system using the Llama-3-Groq Tool-Use models for function calling or API tasks, and a general-purpose model like Llama 3 70B for other language-based requests.

3. Open-Source and Accessible: Both the models are readily available via the Groq API using the model IDs “llama3-groq-70b-8192-tool-use-preview” and “llama3-groq-8b-8192-tool-use-preview.”

AI Search Engine for RAG Apps & AI Agents

Exa is a powerful search engine designed for AI developers. Unlike traditional keyword-based search engines, Exa leverages advanced neural search capabilities and a vast, constantly updated index of high-quality web content. This makes it particularly well-suited for RAG apps for retrieving highly specific content, identifying semantically similar pages, and powering research automation tools.

Exa has announced a major upgrade with its 1.5 release, delivering substantial improvements across its platform.

Key Highlights:

1. Smarter Model - Exa 1.5 is 3x larger than its predecessor and trained with new methods like Matryoshka Representation Learning. It can understand more complex and nuanced search queries to give accurate results, especially when searching for niche information.

2. Expanded Index - Exa 1.5 features an upgraded index with high-value data types, including scientific research papers, company information, news articles, online writing, and even tweets.

3. Hybrid Search (Phrase Filters) - Exa 1.5 introduces hybrid search to combine neural search with keyword matching for highly targeted results. For example, search for “discussions about AI” and filter for mentions of “Elon Musk.”

4. Auto Search with Google Fallback - This intelligent feature automatically determines the best search approach for optimal results. If neural search is insufficient, it defaults to Google keyword search.

5. AI Apps - Exa API is ideal for a range of tasks, including RAG applications. It integrates seamlessly with tools like LangChain, Typescript, OpenAI, CrewAI, and LlamaIndex.

New Cohere Toolkit Feature for Quick Prototyping and Smarter AI Apps 🧠

Cohere Toolkit is an opensource collection of pre-built components for developers to build and deploy RAG applications quickly. Cohere has expanded this toolkit with new features including HTML rendering, configurable authentication, and multi-step tool use for creating sophisticated AI assistants.

Key Highlights:

1. AI-powered HTML Generation - You can now ask Command R models to generate interactive HTML applications directly within the Chat UI. With simple text prompts, the model will generate HTML code for basic web components, such as forms, tables, and layouts.

2. Security with Authentication: You can set up access permissions using email/password authentication, Google OAuth, or OpenID Connect. This ensures secure access to deployed toolkits, especially when dealing with sensitive data sources requiring individual user permissions.

3. Multi-Step Tool Use for Complex Queries: Cohere has integrated its multi-step tool use capability, previously only available via API, into the toolkit. When the model is given a list of tool definitions, it generates a plan of action and decides which tools to use, populates the required parameters, and defines the order of operations.

Quick Bites 🤌

1. London-based company Haiper has released its new video generation model Haiper 1.5 which generates 8-second-long videos from text or image prompts. It can even extend your prior 2 and 4-second videos to 8 seconds, just like Luma Labs Extend feature.
   Below is a comparison of videos generated by Haiper 1.5, Luma Labs, and Runway GEN-3, for the same text prompt.
   ”Dragon-toucan walking through the Serengeti.”

   Not just this, Haiper also has an integrated upscaler that can upscale videos to 1080p in a single click. (Source)
   

2. Menlo Ventures and Anthropic have launched the Anthology Fund with a $100 million fund to invest in early-stage AI companies. The fund will provide startups with $100,000 in funding and $25,000 in credits for using Anthropic’s models. (Source)
   

3. Microsoft’s Designer app is now available on iOS and Android. The app includes features like AI image editing, background removal, and a variety of templates, and integrates with Microsoft apps like Word and PowerPoint. (Source)
   

4. Anthropic has released Claude app for Android. It works just like Claude on iOS and the web. Pick up and continue conversations with Claude across web, iOS, and Android apps. It also supports multimodal inputs, language translation, and advanced reasoning with Claude 3.5 Sonnet. (Source)

Tools of the Trade ⚒️

1. ProctorAI: Monitors your computer screen and alerts you if it detects procrastination. It takes periodic screenshots and analyzes them with multimodal AI models like Claude 3.5 Sonnet or GPT-4o to ensure you’re focused on your tasks.

2. langgraph_streamlit_codeassistant: AI assistant that integrates Python execution capabilities with React component rendering on the fly, offering a comprehensive environment for data analysis, visualization, and interactive web development.

3. Pieces: On-device AI coding assistant that provides contextual solutions to complex development tasks and manages workflow information. It processes data offline, integrates with popular tools, and uses advanced LLMs to generate code and explain concepts.

4. Awesome LLM Apps: Build awesome LLM apps using RAG for interacting with data sources like GitHub, Gmail, PDFs, and YouTube videos through simple texts. These apps will let you retrieve information, engage in chat, and extract insights directly from content on these platforms.

Hot Takes 🔥

1. the biggest barrier to fine tuning LLMs is not cost or modeling or systems expertise anymore, but in collecting high-quality data. it’s hard to do for custom tasks. people try to use gpt 4 as a data generator, which seems ok at a glance but is full of random mistakes at scale ~
   Shreya Shankar
   

2. A lot of SF schizos are just upset that th new X does not prioritize SF schizo content. Discovering that the world doesn’t revolve around you and your niche neurosis can be pretty unsettling. ~
   Bojan Tunguz

Meme of the Day 🤡

😅 Math Olympiad becomes easier for AI; Common sense is still hard.

That’s all for today! See you tomorrow with more such AI-filled content.

Real-time AI Updates 🚨

⚡️ Follow me on Twitter @Saboo_Shubham for lightning-fast AI updates and never miss what’s trending!

PS: I curate this AI newsletter every day for FREE, your support is what keeps me going. If you find value in what you read, share it with your friends by clicking the share button below!

Share Unwind AI

1. Microsoft’s SpreadsheetLLM helps LLMs to process spreadsheets efficiently

2. Mistral AI releases two models specializing in code generation and math

3. Andrej Karpathy launches an AI + education company called Eureka Labs

4. AI software engineer using Claude 3.5 Sonnet with agentic capabilities

& so much more!

Read time: 5 mins

Latest Developments 🌍

Microsoft Lets AI Excel at Large Excel Sheets ⇉⇈

Spreadsheets are the fodder for data science but pose unique challenges in LLM processing. Their expansive grids usually exceed the token limits of LLMs, and the 2D layouts are not suitable for LLM inputs. Furthermore, spreadsheet-specific features such as cell addresses and formats complicate it more LLMs.

Microsoft has introduced SpreadsheetLLM to make spreadsheets more manageable for LLMs. This includes SheetCompressor, which compresses spreadsheets to reduce token usage by a staggering 96%. This tool helps LLMs understand spreadsheet layouts better and improves their performance in various tasks.

Key Highlights:

1. Efficient Encoding - SheetCompressor compresses spreadsheet data to fit within LLMs’ token limits. It uses structural anchors to identify key rows and columns, removing redundant data to create a condensed spreadsheet.

2. Improved Performance: It significantly improves performance in spreadsheet table detection tasks, outperforming the vanilla approach by 25.6% in GPT4’s in-context learning setting.

3. Reduced Token Usage: SheetCompressor optimizes token usage by employing an inverted-index translation method that indexes non-empty cell texts and merges identical values, reducing tokens needed by 96%.

4. Enhanced Accuracy: Fine-tuned LLMs with SheetCompressor achieve a compression ratio of 25x and a state-of-the-art 78.9% F1 score, surpassing previous best models by 12.3%.

Why it matters: Professions like data science and finance rely heavily on spreadsheet applications like Sheets and Excel. However, AI assistance in these applications is limited to generating formulas and charts, mostly for spreadsheets with a few cells. This compression method paves way for mainstream use of LLMs in data analysis tasks to handle larger and more complex spreadsheets. 

AI Platform for Surfacing Real Customer Pains to Prevent Churn

To create a successful business, understanding customer feedback is the most crucial yet challenging part. Traditional metrics like NPS and CSAT don’t provide the full picture as they are too simplistic and cannot capture the nuances of customer sentiment.

Here is Syncly (YC W23), the AI-powered customer feedback analysis tool to help you get deep insights into your CX. It uses AI to analyze everyday customer interactions and give a complete view of customer sentiment. Syncly seamlessly integrates with multiple communication platforms, automatically categorizing feedback and providing actionable insights.

Key Highlights:

1. Seamless integration - Syncly can integrate seamlessly with your favorite tools (Zendesk, Gorgias, Intercom, Gong, Front, etc.)

2. Holistic Visibility - With Smart Insight, identify the most critical issues that hinder customer experience without having to read all customer interactions across the user journey.

3. Actionable Insights - You can Ask Syncly to pinpoint the positive and negative sentiment drivers and get instant action items to elevate customer experience.

4. Sentiment Monitoring at Every Stage - With Customer AI, you can monitor sentiment trends by account and by individual user, to proactively engage with customers with underlying issues before they become at-risk.

5. Cross-Functional Collaboration - The platform provides an easy way to create and share relevant analyses, charts, graphs, and action items across the organization, breaking down silos and promoting data-driven decision-making.

Try Syncly for free and start elevating your customer experience today!

New Specialized Models by Mistral AI 🤖

Mistral AI has released two new specialized models: Codestral Mamba which specializes in code generation, and Mathstral designed for math reasoning and scientific discovery.

Codestral Mamba

A Mamba2 language model specialized in code generation. Mamba models offer linear time inference so you can engage with the model extensively with quick responses, irrespective of the input length.

1. It outperforms other 7B models on various coding benchmarks and even competes with larger models including Mistral’s Codestral 22B and Meta’s CodeLlama 34B.

2. You can deploy Codestral Mamba using the mistral-inference SDK, or through TensorRT-LLM. For local inference, it should soon be available on llama.cpp. Raw weights are available on HuggingFace.

Mathstral

This is Mistral’s first Mathstral model with 7B parameters, designed for advanced mathematical problems requiring complex, multi-step logical reasoning. The model has a 32k context window published under the Apache 2.0 license.

1. Based on Mistral 7B model, it specializes in STEM subjects. It achieves SOTA reasoning capacities in its size category across various benchmarks. It scored 56.6% on MATH and 63.47% on MMLU.

2. Mathstral scored 68.37% on MATH with majority voting and 74.59% using a strong reward model among 64 candidates, demonstrating significant improvement with more inference-time computation.

3. Mathstral can be fine-tuned using mistral-finetune, with weights available on HuggingFace.

Quick Bites 🤌

1. Andrej Karpathy is starting an AI+Education company called Eureka Labs with the goal of creating an AI-native school. Partnering with subject matter experts, the first product will be LLM101n, an undergraduate-level course guiding students to train their own AI. (Source)

2. Tech giants Apple, Nvidia, and Anthropic have used over 173,000 YouTube videos to train AI models without creators’ consent, according to an investigation by Proof News. These include popular educational and creators channels like Marques Brownlee and Khan Academy. (Source)

3. YouTube is rolling out a new feature in YT Music called “sound search” which is an advanced version of Shazam. It lets you hum, sing, or play audio to search and recognize the music.

   

   YouTube is also testing an “AI-generated conversational radio” in the US for Premium users. It lets you create a custom radio by “describing exactly what you want to hear.” (Source)

4. Anthropic has doubled the max output token limit for Claude 3.5 Sonnet from 4096 to 8192 in the Anthropic API. Just add the header “anthropic-beta”: “max-tokens-3-5-sonnet-2024-07-15” to your API calls. (Source)

5. The UK’s Competition and Markets Authority (CMA) is launching an antitrust probe into Microsoft’s investment in Inflection AI, focused mostly on hiring Inflection’s staff. (Source)

Tools of the Trade ⚒️

1. Claude Engineer 2.0: Claude Engineer with AI agents is a CLI tool that combines the capabilities of Claude 3.5 Sonnet with practical file system operations, web search, vision support and intelligent code execution to build apps. It can do the following and so much more:

   ✅ Create a new Python project structure for a web application

   ✅ Explain the code in file.py and suggest improvements

   ✅ Search for the latest best practices in React development

   ✅ Help me debug this error: [paste your error message]

   ✅ Analyze this image and describe its contents

2. PeachML: AI marketplace where you can create a model, deploy it for free, and earn money whenever others use it. Train in your preferred environment, deploy with a click, and get paid per inference used.

3. Financial Datasets: This stock market API provides 30+ years of financial data for all S&P 500 companies, including income statements, balance sheets, and cash flow statements. Connect your AI financial agents for analysis, portfolio management, or trading with no API limits during the open beta.

4. Awesome LLM Apps: Build awesome LLM apps using RAG for interacting with data sources like GitHub, Gmail, PDFs, and YouTube videos through simple texts. These apps will let you retrieve information, engage in chat, and extract insights directly from content on these platforms.

Hot Takes 🔥

1. ….I wished for tech to look like "a thriving coral reef" ecosystem but sometimes it feels more like mostly plankton, a few clown fish, two tunas, and 5 killer whales circling above. ~
   Andrej Karpathy
   

2. LLMs attracted way too people who don't understand the basics of building systems with non-deterministic components to machine learning. ~
   Jaana Dogan

Meme of the Day 🤡

That’s all for today! See you tomorrow with more such AI-filled content.

Real-time AI Updates 🚨

⚡️ Follow me on Twitter @Saboo_Shubham for lightning-fast AI updates and never miss what’s trending!

PS: I curate this AI newsletter every day for FREE, your support is what keeps me going. If you find value in what you read, share it with your friends by clicking the share button below!

Share Unwind AI

1. Microsoft builds a data flywheel for LLMs post-training via Simulated Chatbot Arena

2. Fireworks AI launches FireAttention V2 for 12x faster inference on long context

3. Google Gemini might be scanning your files from Drive, secretly

4. Mem0 - The memory layer for Personalized AI

& so much more!

Read time: 3 mins

Latest Developments 🌍

The Data Flywheel for Chatbots 📝

Platforms like the LMSYS Chatbot Arena, which pits different chatbots against each other and uses human feedback to rank them, are valuable tools for improving LLM performance post-training. But it often involves resource-intensive and time-consuming human evaluations, creating a bottleneck in the speed and scale of LLM improvement.

To address this, Microsoft proposes “Arena Learning,” a training and evaluation pipeline fully based on and powered by LLMs. Arena Learning simulates these chatbot competitions using AI, eliminating the need for human involvement in the evaluation process. This simulated arena enables a continuous feedback loop for ongoing model improvement.

Key Highlights:

1. Automated Evaluation - Arena Learning employs an “AI judge” to assess simulated chatbot battles, mimicking human judgment and providing rankings, scores, and explanations for each interaction.

2. Iterative Improvement - The target LLM is repeatedly pitted against other state-of-the-art models, with each iteration generating data used to refine its capabilities through various training techniques.

3. Accurate Prediction - It demonstrates high consistency with the human-evaluated LMSYS Chatbot Arena, effectively predicting model rankings in a controlled offline environment.

4. Performance Gains - LLMs trained using Arena Learning show considerable improvement across multiple evaluation metrics, including Elo rankings, win rates, and benchmark scores.

Why it matters: Arena Learning could drastically reduce the time and cost associated with model evaluation so you can focus more on innovation and less on tedious processes. Removing the need for human annotators, it can scale your evaluations more effectively.

FireAttention V2 to Optimize High-Traffic AI Applications 📈

Fireworks AI previously released FireAttention, a specialized CUDA kernel within their proprietary LLM serving stack to speed up the serving of open-source models like Mixtral. Focusing on use cases with 1K token prompt and 50 generated tokens, it demonstrated a 4x speed increase compared to alternatives like vLLM, while maintaining comparable accuracy.

Now, Fireworks AI has released FireAttention V2 to tackle long context (8K-32K) inference. V2 incorporates specific optimizations for the Hopper architecture and a multi-host deployment mode for high-traffic scenarios. This resulted in up to 12x faster inference times compared to previous versions.

Key Highlights:

1. Beyond Needle in a Haystack - Fireworks argues that NIAH isn’t a sufficient benchmark for true long-context understanding. Instead, they utilize the RULER benchmarking library for multi-needle, variable tracking, and question-answering tasks.

2. Qwen 72B Excels - Fireworks found that among open-source models, Qwen 72B demonstrates the strongest performance on these more complex long context benchmarks.

3. Dominating Performance Across Contexts - FireAttention V2 consistently outperforms vLLM in both short-medium and long-generation tasks, showing high throughput and latency advantages of 1.7x to 8x depending on the specific configuration.

4. Multi-host Deployment for Better Throughput - FireAttention multi-host mode allows FireAttention to improve throughput by ~2x more while keeping latency improvement relatively intact.

5. Try now - Qwen 2 72b Instruct, Mixtral 8x7B Instruct and many other models are available for you to try right now on fireworks.ai and can produce with the abovementioned speed.

Why it matters: FireAttention V2 significantly improves LLM inference for high-traffic and long-context scenarios, improving user experiences. The focus on long-context tasks also pushes LLM evaluation beyond simple retrieval, better reflecting real-world application needs. As the scope of RAG and multi-agent applications increases, optimizations like this will become crucial for maintaining performance and efficiency.

Quick Bites 🤌

1. Amazon’s AI shopping assistant Rufus is now available for all Amazon customers in the US. Living in the bottom of the app, Rufus can answer your questions about product details, discovering and covering options, current and past orders, and even non-shopping related questions.

   

2. HP has introduced HP OmniBook Ultra, the world’s highest-performance AI PC with up to an industry-leading 55 TOPS of NPU performance. Powered by the newly launched AMD Ryzen AI 300 series processor especially designed for Copilot + PCs, it offers unparalleled performance and up to 21 hours of battery life.

3. Google’s Gemini has been scanning PDF files in Google Drive without user permission. An X user Kevin Bankston shared on the platform that Gemini automatically summarized his tax return in Google Docs without prompting it. The surprising part is that his Google Workspace extension is turned off in the settings but Gemini continues to scan his documents without permission.

Tools of the Trade ⚒️

1. Mem0: Create personalized AI experiences by retaining information across sessions and continuously improving based on user interactions. It provides a straightforward API for easy integration into various applications for consistent and adaptive personalization.

2. AI-Renamer: A Node.js CLI tool that renames files based on their contents using Ollama and LM Studio models like Llava and Llama. It extracts frames from videos with ffmpeg and uses the models to rename the files.

3. Booth.ai: No-code generative AI app builder to create powerful AI solutions in minutes. You can build custom tools for tasks like categorizing customer inquiries, analyzing PDFs, or automating tasks, using its library of 165+ nodes and various AI models.

4. Awesome LLM Apps: Build awesome LLM apps using RAG for interacting with data sources like GitHub, Gmail, PDFs, and YouTube videos through simple texts. These apps will let you retrieve information, engage in chat, and extract insights directly from content on these platforms.

Hot Takes 🔥

1. Synthetic data is dumb, no shot you get better models from it. Have you ever read the synthetic instructions? I never ask questions like that to LLMs ~
   anton
   

2. You either die a hero or live long enough to become the next IBM. ~
   Bojan Tunguz

Meme of the Day 🤡

That’s all for today! See you tomorrow with more such AI-filled content.

Real-time AI Updates 🚨

⚡️ Follow me on Twitter @Saboo_Shubham for lightning-fast AI updates and never miss what’s trending!

PS: I curate this AI newsletter every day for FREE, your support is what keeps me going. If you find value in what you read, share it with your friends by clicking the share button below!

Share Unwind AI

1. Train AI models globally using peer-to-peer network with this opensource framework

2. Build RAG apps that can process 10 Million words

3. Meta is releasing Llama-3 405B on July 23

4. OpenAI is reportedly working on an AI project codenamed Strawberry

& so much more!

Read time: 3 mins

Latest Developments 🌍

Open Framework for Distributed Model Training 🌐

Training LLMs is a resource-intensive process, often requiring access to massive, centralized computing power. This has created a barrier to entry for many developers and researchers. Prime Intellect, a US-based startup that provides a platform for finding global compute resources, has released an open-source framework for enabling globally distributed LLM training.

OpenDiLoCo enables decentralized LLM training by using a peer-to-peer network for communication, even with limited internet bandwidth. It has been tested with models spread across two continents and three countries while maintaining 90-95% compute utilization.

Key Highlights:

1. How it works - OpenDiLoCo builds upon Google DeepMind’s Distributed Low-Communication (DiLoCo) method for data-parallel training of language models on islands of devices that are poorly connected. Instead of synchronizing training data after every step, it only shares updates every 500 steps, drastically reducing the amount of data that needs to be transferred between machines.

2. DiLoCo Scaled - OpenDiLoCo successfully scales the DiLoCo method to handle models with up to 1.1 billion parameters, compared to the original 400 million parameters in DeepMind’s work.

3. Global Decentralized Training - OpenDiLoCo supports training across continents and countries, achieving 90-95% compute utilization. It uses a peer-to-peer network with Hivemind for inter-node communication and PyTorch FSDP for intra-node communication.

4. Fault Tolerance - OpenDiLoCo allows on/off ramping of resources and maintains training even if some devices become unavailable. Hivemind’s fault-tolerant training ensures that training continues smoothly.

5. Open Source - It is developed on the Hivemind library and the codebase is publicly accessible. Training can be carried out with as little as two GPUs, which don’t need to be co-located.

Chat Apps with Built-in RAG and Explainable AI 🧠

Writer, an AI startup that provides a full-stack generative AI platform, released its AI Studio last month. With this Studio, enterprises can quickly build and deploy a wide range of AI apps, including text generation apps, chat apps, and more, all without any coding.

They have released some major upgrades to their platform. These upgrades include built-in RAG to analyze up to 10 million words, explainable AI features, dedicated modes, voice rewrites, custom instructions, and more to make it more powerful and easy to use.

Key Highlights:

1. 10 Million-Word Analysis - With chat apps, you can now upload files with up to 10 million words. Writer uses graph-based RAG where uploaded files are broken down into data points and their semantic relationships are mapped in a graph structure. When you ask questions, the AI retrieves relevant data points from the graph and uses them to generate accurate and contextually relevant responses.

2. Explainable AI for Transparency - Writer’s chat apps now feature “thought process” functionality. The AI breaks down complex questions into sub-questions, providing answers and citing sources for each. This transparency can help you refine your prompts for better results.

3. Dedicated Modes for Specific Tasks - Recognizing that a one-size-fits-all approach isn't optimal, Writer introduces “modes” – dedicated user experiences tailored for different tasks. You can choose between

   1. General mode for on-the-fly assistance on general knowledge,

   2. Document mode for deep dives into a few specific files, and

   3. Knowledge Graph mode for accessing company knowledge bases and getting answers based on large corpora of data.

4. Customization - Additional features enhance customization and user experience. These include voice profile rewriting, custom instructions that apply automatically to outputs, and dictation capabilities.

Quick Bites 🤌

1. Meta is planning to release the largest Llama-3 model with 405B parameters on July 23. As Meta had announced earlier, Llama-3 405B will be multimodal, multilingual, and have a large context window. Its performance had reached GPT-4 level while it was in training. It’d be the most capable opensource model to date! (Source)

   

2. Some updates from OpenAI

   1. OpenAI is reportedly working on an AI project codenamed Strawberry. Strawberry models can not just generate answers to queries but also plan ahead enough to navigate the internet autonomously and reliably to perform what OpenAI terms “deep research.” (Source)

   2. OpenAI has come up with a set of five levels to track its progress toward building AGI capable of outperforming humans. These tiers range from AI with conversation abilties to that can do the work of an organization. OpenAI claims that it is currently on Level 1 and is at the cusp of reaching Level 2. (Source)

      

   3. OpenAI landed in controversy in May when employees called out the company for making them sign NDAs with lifetime nondisclosure and non-disparagement clauses. The whistleblowers have reached out to the US financial watchdog to investigate these NDAs, claiming that these NDAs violate SEC rules by waiving employees’ federal whistleblower rights. (Source)
      

3. A bipartisan group of senators introduced the COPIED Act to protect artists, songwriters, and journalists from having their work used for AI training without consent. The bill requires AI companies to provide content provenance information so the creators can control and set terms for their work. It also mandates guidelines to identify AI-generated and altered content. (Source)

Tools of the Trade ⚒️

1. Highlight: Access ChatGPT, Claude 3.5, and Perplexity in one single interface, while staying in the flow. It appears on any website or app with just a shortcut (Command +.). You can type in your query, take a screenshot and paste it, or just ask it to take a screenshot of what you’re looking at.

2. Corgi: The first AI insurance company for small/mid-sized businesses to seamlessly buy and manage customized insurance policies tailored to their needs. It uses AI to handle applications, provide clear policy explanations, and offer instant support.

3. Tribe AI: Low-code tool to build and manage multi-agent teams to handle complex tasks efficiently. It lets you customize agent roles, maintain persistent conversations, monitor performance in real time, and easily deploy using Docker.

4. Awesome LLM Apps: Build awesome LLM apps using RAG for interacting with data sources like GitHub, Gmail, PDFs, and YouTube videos through simple texts. These apps will let you retrieve information, engage in chat, and extract insights directly from content on these platforms.

Hot Takes 🔥

1. One more reason to pick an AI President - at a minimum it can’t be assassinated ~
   Bindu Reddy
   

2. AI will increase the need for human intelligence, not decrease it. ~
   Pedro Domingos

Meme of the Day 🤡

That’s all for today! See you tomorrow with more such AI-filled content.

Real-time AI Updates 🚨

⚡️ Follow me on Twitter @Saboo_Shubham for lightning-fast AI updates and never miss what’s trending!

PS: I curate this AI newsletter every day for FREE, your support is what keeps me going. If you find value in what you read, share it with your friends by clicking the share button below!

Share Unwind AI

Here are 10 AI breakthroughs that you can’t afford to miss 🧵👇

10x Faster Inference for 1M+ Context LLMs 🐎

Microsoft researchers developed MInference, a new method to address the computational bottleneck in LLM inference with long prompts. It analyzes and leverages patterns in how LLMs pay attention to text, making the whole process much faster without compromising the LLM’s accuracy.

MInference makes pre-fill up to 10x faster for 1 million token prompts on a single A100 GPU. Tasks that previously took half an hour might now only take a few minutes.

China’s Multimodal AI Model Outshines GPT-4o and Claude 3.5 🌟

At the World Artificial Intelligence Conference, Chinese AI company SenseTime introduced its new multimodal AI model SenseNova 5o and the improved language model SenseNova 5.5. SenseNova 5o can process text, images, audio, and video, and is suitable for real-time interactions. The model outperforms state-of-the-art models like GPT-4o and Claude 3.5 Sonnet across most of the standard benchmarks.

YouTube’s AI Tool to Remove Copyrighted Music 🎧

YouTube is updating its “Erase Song” feature with a new AI feature that will identify and remove copyrighted music from videos while keeping other audio elements intact. Creators can choose between “Erase Song” to target only the copyrighted audio or “Mute All Sound” to silence everything within specified timestamps.

Build and Deploy Multi-Agent Workflows Without Code

Microsoft released AutoGen Studio, a low-code interface to create multi-agent AI applications. This new tool builds on the existing AutoGen framework for developers to build, test, and share AI agents and workflows. With AutoGen Studio, developers can create complex agent interactions with minimal coding. The goal is to make building and deploying these sophisticated AI systems accessible to everyone.

Create, Publish, Share and Even Remix Artifacts 🎨

Artifacts that appears in Claude.ai can now be published and shared with others. Not just this, you can take an Artifact shared with you in a new chat with Claude and remix it with your own unique spin!

Samsung Brings New Generative AI Features to its Device 🪄

Samsung wrapped its Unpacked Event 2024 yesterday, revealing the next generations of Z Fold and Flip phones, Galaxy buds, watch, and the highly-anticipated Samsung Ring. Throughout the event, AI took centre stage integrating into every product through Galaxy AI powered by Google’s Gemini AI model.

• Samsung Health app is now powered by Galaxy AI to track vitals and interpret them to give you actionable insights.

• The new AI-powered Galaxy Ring is designed to be worn 24/7 to track your health and vitals throughout day and night.

• New Generative AI features in Galaxy AI include Composer to draft emails and messages, transcription and summarizing audio notes, sketch-to-image, Live Translate in 16 different languages, and more.

Evaluate Prompts in Anthropic’s Developer Console 📝

Prompt quality is crucial for AI application results. Anthropic Console now offers tools to generate, test, and evaluate prompts within a streamlined workflow, eliminating the need for manual testing.

It leverages Claude 3.5 Sonnet to generate effective prompts and test cases. You can create and manage extensive test suites, compare multiple prompt versions, and even have experts grade response quality on a 5-point scale.

Microsoft and Apple Drop Observer from OpenAI’s Board 🧑🏻‍💼

Microsoft has relinquished its observer seat on OpenAI’s board, a position it held for less than eight months. Apple too was initially slated to appoint an observer on OpenAI’s board, but has also opted out. OpenAI will now engage with Microsoft and Apple through regular stakeholder meetings.

AWS’ AI-Powered Low-Code Studio for Creating Apps 🧑‍💻

AWS has introduced AWS App Studio, currently in public preview, for users to build enterprise-grade custom applications quickly, using natural language instead of traditional coding.

Just describe your desired application in plain language, and App Studio’s AI will generate the core components, including the UI, data structures, and basic logic. It also has pre-built connectors for services like Amazon Aurora, DynamoDB, S3, and Salesforce to integrate data for applications.

Opensource Framework for Cost-Effective LLM Routing ↖️↗️

LMSYS has developed RouteLLM, an open-source framework for cost-effective LLM routing. This system acts as an intelligent gatekeeper, analyzing incoming queries and intelligently directing them to the most appropriate LLM based on both the task’s complexity and the capabilities of available models.

RouteLLM achieved cost savings of over 85% on the MT Bench, 45% on MMLU, and 35% on GSM8K compared to using only GPT-4, all while maintaining 95% of GPT-4’s performance.

Which of the above AI development you are most excited about and why?
Tell us in the comments below ⬇️

That’s all for today 👋

Stay tuned for another week of innovation and discovery as AI continues to evolve at a staggering pace. Don’t miss out on the developments – join us next week for more insights into the AI revolution!

Click on the subscribe button and be part of the future, today!

Subscribe now

📣 Spread the Word: Think your friends and colleagues should be in the know? Click the ‘Share’ button and let them join this exciting adventure into the world of AI. Sharing knowledge is the first step towards innovation!

Share

🔗 Stay Connected: Follow us for AI updates, sneak peeks, and more. Your journey into the future of AI starts here!

Shubham Saboo - Twitter | LinkedIn ⎸ Unwind AI - Twitter | LinkedIn

Awesome LLM Apps | Sponsor Us

1. Build enterprise-grade applications with AWS’s Low-Code Generative AI studio

2. LMSYS open framework for LLM routing that saves 85% cost with 95% GPT-4’s performance

3. Build, deploy, and operate fully custom AI Copilots with this opensource kit

4. Turn any website into an API with AI with SmartCrawl

& so much more!

Read time: 3 mins

Latest Developments 🌍

AWS’ AI-Powered Low-Code Studio for Creating Apps 🧑‍💻

Amazon Web Services (AWS) has introduced AWS App Studio, currently in public preview, for users to build enterprise-grade custom applications quickly, using natural language instead of traditional coding. App Studio offers a streamlined process to create functional applications with user interfaces, data models, and business logic in minutes.

Key Highlights:

1. Generative AI for App - Just describe your desired application in plain language, and App Studio’s AI will generate the core components, including the UI, data structures, and basic logic.

2. Built-in Connectors - The Studio offers pre-built connectors for services like Amazon Aurora, DynamoDB, S3, and Salesforce to integrate data for applications.

3. Visual Building Environment - You can refine your applications with a simple drag-and-drop interface for UI design, and easily define workflows and automations.

4. Role-Based Access Control - App Studio integrates with AWS IAM Identity Center for secure user management so the administrators can define granular permissions for different user groups.

5. Managed Infrastructure - AWS App Studio handles application deployment and operations. You can also test it before pushing it to production.

Opensource Framework for Cost-Effective LLM Routing ↖️↗️

Deploying LLMs in real-world applications often presents a dilemma: more capable models deliver higher-quality results but come at a higher cost, while smaller models are more budget-friendly but may compromise performance.

LMSYS has developed RouteLLM, an open-source framework for cost-effective LLM routing. This system acts as an intelligent gatekeeper, analyzing incoming queries and intelligently directing them to the most appropriate LLM based on both the task’s complexity and the capabilities of available models.

Key Highlights:

1. Preference Data-Driven - RouteLLM is trained on preference data which allows it to learn the strengths and weaknesses of different LLMs and match them to specific types of queries.

2. Data Augmentation Techniques - It incorporates data augmentation methods that further improve routing accuracy, even with limited training data.

3. Cost Reductions - In tests, RouteLLM achieved cost savings of over 85% on the MT Bench, 45% on MMLU, and 35% on GSM8K compared to using only GPT-4, all while maintaining 95% of GPT-4’s performance.

4. Open-Source and Available - RouteLLM, along with its datasets and trained routers, is publicly available on GitHub and HuggingFace.

Quick Bites 🤌

1. OpenAI’s text-to-speech API is now integrated into their Playground, so you can easily generate audio from text using six voices. You can try it here! (Source)

2. The Washington Post has released a new AI chatbot “Climate Answers” that answers questions about climate change and related topics, powered by OpenAI’s LLM. It generates responses using information from The Washington Post’s climate section articles dating back to 2016 and cites its sources. (Source)

3. AMD is acquiring Silo AI, Europe’s largest private AI lab, for $665 million to boost its enterprise AI solutions. This acquisition will allow AMD to integrate Silo AI's expertise in developing AI models and solutions. (Source)

4. Elon Musk’s brain-computer interface company, Neuralink, shared an update on X about its device. Key takeaways:

   • Long-Term Goals - Neuralink aims to mitigate AI risks, treat brain and spinal cord injuries, and restore sight to the blind.

   • First Patient - Neuralink’s first human patient is using the device to control a computer with his thoughts, breaking world records for cursor control speed while enjoying games and internet browsing.

   • Future Research - Neuralink is exploring ways to restore mobility using the device to control wheelchairs and robotic limbs. Research also includes treating mental health conditions like schizophrenia, memory issues, and expanding our understanding of the brain.

Tools of the Trade ⚒️

1. CopilotKit: Build and deploy AI copilots into your applications, such as chatbots, AI assistants, and AI-enhanced text areas, using simple components and APIs. You can customize these features to interact with your application’s data and perform actions.

2. Viral Factory: Automatically creates various social media content formats like videos and scripts. You can customize its production process in detail, including text-to-speech, background visuals, and uploading to platforms like TikTok.

3. SmartCrawl: Turn any website into an API with AI. Extract structured data from any website and access it via API. You can use this data to automate tasks or integrate it into your apps.

4. Awesome LLM Apps: Build awesome LLM apps using RAG for interacting with data sources like GitHub, Gmail, PDFs, and YouTube videos through simple texts. These apps will let you retrieve information, engage in chat, and extract insights directly from content on these platforms.

Hot Takes 🔥

1. Computers will control the future
   …
   …
   …
   They already do ~
   Elon Musk
   

2. The biggest puzzle in AI is why the dozens of companies that raised billions in funding can’t catch up to Sonnet 3.5 🤷‍♀️🤷‍♀️ ~
   Bindu Reddy

Meme of the Day 🤡

That’s all for today! See you tomorrow with more such AI-filled content.

Real-time AI Updates 🚨

⚡️ Follow me on Twitter @Saboo_Shubham for lightning-fast AI updates and never miss what’s trending!

PS: I curate this AI newsletter every day for FREE, your support is what keeps me going. If you find value in what you read, share it with your friends by clicking the share button below!

Share Unwind AI

1. Samsung releases new generative AI features powered by Google’s Gemini

2. Generate, test, and evaluate prompts in Anthropic’s Developer Console

3. Microsoft and Apple drop their observer from OpenAI’s board

4. First-ever opensource LLM designed for the semiconductor industry

5. Use Claude’s Artifacts with GPT-4o, Gemini 1.5 Pro, Llama 3 and other AI models

& so much more!

Read time: 3 mins

Latest Developments 🌍

Samsung Brings New Generative AI Features to its Device 🪄

Samsung wrapped its Unpacked Event 2024 yesterday, revealing the next generations of Z Fold and Flip phones, Galaxy buds, watch, and the highly-anticipated Samsung Ring. Throughout the event, AI took centre stage integrating into every product through Galaxy AI powered by Google’s Gemini AI model. Here’s a bite of everything AI that was announced:

1. Samsung Health app is now powered by Galaxy AI to not just track vitals like sleep, heart rate, and activity levels but also interpret them to give you meaningful and actionable insights. For eg., a new index called Energy Score will give you a personalized snapshot of your body’s readiness for the day when you wake up, and suggests if you’re well rested, should balance, or need to take it easy.

2. Galaxy Ring: Call it the AI wearable for health. Priced at $399, it is designed to be worn 24/7 to track your health and vitals throughout day and night and feed the information to Samsung Health to give you personalized insights.

3. Galaxy Watch Ultra: Designed for athletes and fitness enthusiasts, it is powered by WatchOS in partnership with Google to integrate AI. It can help plan your workouts and trainings strategically. It also has the first-of-its-kind sensors for detecting signs of sleep apnea, approved by the US FDA.

4. Galaxy Buds3 Pro: Besides the better sound clarity, these buds understand if you’re wearing the buds loosely or tightly, and adjust the sound waves accordingly to give clear and immersive audio.

5. New Generative AI features in Galaxy AI in the new Galaxy Z6 series phones:

   1. It enables multimodal interactions, powered by Google’s Gemini AI model. Be it Circle to Search, translating text in an image, solving math problems, or asking questions about YouTube videos.

   2. Composer is your AI writing assistant with contextual relevance. Give it a text or a voice prompt and it’ll draft your email or message replies.

   3. Samsung Notes now has a voice recorder. You can record, and Galaxy AI can transcribe and summarize it. Transcription can separate up to 10 individual speakers.

   4. Sketch-to-Image - When you sketch on a blank canvas, Galaxy AI understands what you want to sketch and it’ll generate images in different styles. This also works in Photos where you can sketch within images and embed real-life objects in it.

   5. Photo Assist in your gallery can identify portraits and recommend how you can imagine them in different styles.

   6. Live Translate designed primarily for phone calls, can now translate 16 languages on-device, without internet. It will also be available in popular messaging and calling third-party apps. Interpreter designed for real-time face-to-face translation will also power live voice-to-voice translation with Galaxy Buds.

   7. Hybrid approach - Some features like Live Translate and Interpreter are handled on-device. On-device AI builds a knowledge graph for each user, learning from your usage over time to provide tailored services. Other generative AI features that require intensive processing are done on the cloud. Galaxy AI also gives you the option to choose which features you want to be processed on-device only.

6. New AI features for Camera and Wallpaper in Galaxy Z Flip 6:

   1. Camera in the phone uses AI to detect objects, identify lighting conditions, isolate delicate facial features to enhance and sharpen them, and make the image more natural.

   2. When capturing images hands-free, Auto-Zoom finds the best scene for your shot by detecting the subject and making necessary adjustments to capture the perfect picture without your involvement.

   3. AI makes wallpapers interactive and even adaptive. The wallpapers automatically change the brightness and background according to the lighting conditions around you and even the weather.

Evaluate Prompts in Anthropic’s Developer Console 📝

Prompt quality significantly impacts the results of your AI applications. To help you craft the perfect prompts and speed the development process, Anthropic Console now offers features to generate, test, and evaluate prompts within a streamlined workflow. This eliminates manual testing and comparison across different platforms.

Key Highlights:

1. Prompt Generation and Testing -

   1. Simply describe the task you want your AI application to perform (e.g., “Summarize customer feedback”).

   2. The Console leverages Claude 3.5 Sonnet to generate an effective prompt and automatically creates test cases with relevant input data to evaluate its performance.

2. Test Suite -

   1. Create and manage extensive test suites in the Console. You can manually input test cases, import them in bulk, or ask Claude to auto-generate test cases for you with the ‘Generate Test Case’ feature.

   2. You can even view and adjust Claude’s understanding of the generation requirements for each variable.

3. Compare and Iterate Prompts -

   1. Create multiple prompt versions, execute the entire test suite with a single click, and directly compare outputs side-by-side to see the impact.

   2. You can also have subject matter experts grade response quality on a 5-point scale to see whether the changes you’ve made have improved response quality.

Quick Bites 🤌

1. Microsoft has relinquished its observer seat on OpenAI’s board, a position it held for less than eight months. Apple too was initially slated to appoint an observer on OpenAI’s board, but has also opted out. OpenAI will now engage with Microsoft and Apple through regular stakeholder meetings. (Source)

2. OpenAI has partnered with one of the United States’ leading national laboratories, Los Alamos National Laboratory, to study how AI can be used safely by scientists conducting bioscience research. The focus is to evaluate how AI models with multimodal capabilities can assist scientists with laboratory tasks. (Source)

3. US-based company Aitomatic which builds autonomous AI agents for industries, has introduced SemiKong, the first-ever opensource LLM specifically designed for the semiconductor industry. It has greater accuracy and relevance than generic LLMs like GPT and Llama 3 on industry-specific tasks. It is available for download on HuggingFace and GitHub. (Source)

Tools of the Trade ⚒️

1. Artifacts with other LLMs: Claude Sonnet 3.5 Artifacts is now available to use with GPT-4o, Gemini, and Llama-3. Abacus AI’s ChatLLM playground gives access to all leading AI models in one place for just $10 a month, with no usage cap and artifacts with them.

2. Julius AI Workflows: Execute complex, multi-step data analyses using pre-built AI-powered templates to automate the entire flow. Streamline data analysis tasks like visualization, statistical testing, forecasting, and more without any coding.

3. Reworkd AI: A web scraping tool that automatically extracts and structures data from websites according to your specifications. You can use it to collect large amounts of data from multiple websites without writing code.

4. Awesome LLM Apps: Build awesome LLM apps using RAG for interacting with data sources like GitHub, Gmail, PDFs, and YouTube videos through simple texts. These apps will let you retrieve information, engage in chat, and extract insights directly from content on these platforms.

Hot Takes 🔥

1. I know it's become fashionable to go "gen AI is a bubble" and "where's the revenue" with Goldman Sachs and Sequoia and more, but like GPT-4 came out barely a year ago and we've barely unwrapped the new data centers, so can y'all just give it a minute!? ~
   Rohit Krishnan

2. You don't imprison a baby because he might grow into a criminal, but that's what lawmakers are trying to do with AI. ~
   Pedro Domingos

Meme of the Day 🤡

That’s all for today! See you tomorrow with more such AI-filled content.

Real-time AI Updates 🚨

⚡️ Follow me on Twitter @Saboo_Shubham for lightning-fast AI updates and never miss what’s trending!

PS: I curate this AI newsletter every day for FREE, your support is what keeps me going. If you find value in what you read, share it with your friends by clicking the share button below!

Share Unwind AI

1. Microsoft’s low-code studio for building multi-agent AI workflows

2. Lost in the Pages: LLMs struggle with book-length text

3. Create, publish, share and even remix Artifacts in Claude

4. OpenAI blocks its AI models in China but developers can use them via Microsoft Azure

5. Quora’s Poe now has an Artifacts-like feature to create custom web apps within the chat

& so much more!

Read time: 3 mins

Latest Developments 🌍

Build, Test, and Deploy Multi-Agent Workflows Easily 👩‍👩‍👧‍👦

Microsoft has released AutoGen Studio, a low-code interface to create multi-agent AI applications. This new tool builds on the existing AutoGen framework for developers to build, test, and share AI agents and workflows. With AutoGen Studio, developers can create complex agent interactions with minimal coding. The goal is to make building and deploying these sophisticated AI systems accessible to everyone.

Key Highlights:

1. Author Agent Workflows - Users can choose from pre-defined agents or create their own, customizing them with different skills, prompts, and foundation models.

2. Debugging and Testing - Developers can test their workflows in real-time and observe the interactions between agents. They can analyze the “inner monologue” of the agents, review generated outputs, and get insights into performance metrics like token usage and API calls.

3. Reusable Components and Deployment - Users can download the skills, agents, and workflow configurations they create, and share and reuse these artifacts.  They can also export workflows as APIs to deploy them in other applications or cloud services.

4. Addressing Real-world Problems - AutoGen Studio is already being used to prototype tasks like market research, structured data extraction, video generation, and more.

Needle-in-a-Haystack in Not All You Need 📌

Current long-context LLMs struggle to understand and reason over information found within book-length texts. While these models may nail basic retrieval tasks like needle-in-the-haystack, they fall short when they need to synthesize information from various parts of a long document.

To address this, researchers have created NOCHA, a dataset of 1,001 true/false claim pairs about fictional books to test an LLM’s ability to comprehend and reason over lengthy narratives. NOCHA requires models to analyze the entire text, including implicit information and nuances specific to the narrative to make it more challenging and realistic. Early results show even the most advanced LLMs struggle with this level of complex reasoning.

Key Highlights:

1. Open-weight models perform poorly - All open-weight LLMs tested performed poorly on NOCHA. In contrast, GPT-4o achieved the highest accuracy at 55.8%, still a long way from perfect understanding.

2. Global reasoning poses a major hurdle - LLMs generally performed better on NOCHA pairs requiring sentence-level retrieval compared to those demanding global reasoning. There is a significant gap in their ability to connect information across large portions of text.

3. Model explanations are often unreliable - Even when the model’s final answer was correct, its explanations for true/false judgments were inaccurate. Models might reach the correct conclusion through faulty reasoning.

4. Speculative fiction proves most challenging - LLMs showed significantly lower accuracy on claims about speculative fiction compared to historical or contemporary settings. Models struggle to grasp complex, fictional worlds and rules unique to the narrative.

Quick Bites 🤌

1. Artifacts that appears in Claude.ai can now be published and shared with others. Not just this, you can take an Artifact shared with you in a new chat with Claude and remix it with your own unique spin! (Source)
   

2. OpenAI is blocking API access to its AI models in China, citing its policy to restrict usage in unsupported regions. But this block doesn’t apply to Microsoft. Despite this decision by OpenAI, Microsoft has confirmed that its Azure OpenAI Service, operating in China through a joint venture, will continue to offer access to AI models for its Chinese customers. (Source)
   

3. AI Pin startup Humane’s executives, Brooke Hartley Moy and Ken Kocienda, have left the company to start an AI fact-checking search company called Infactory, for enterprise clients like newsrooms and research facilities. The platform will source information directly from trusted providers to mitigate the risk of AI hallucinations. The company is currently pre-seed. (Source)
   

4. Quora’s Poe has a new feature “Previews” which lets you create, interact with, and share custom web applications generated directly in chats on Poe. Previews works particularly well with LLMs that excel at coding, including Claude 3.5 Sonnet, GPT-4o, and Gemini 1.5 Pro. You can create custom interactive experiences like games, animations, data visualizations, regardless of programming ability. (Source)

Tools of the Trade ⚒️

1. Ollama 0.2: This version comes with two new features. It lets you handle multiple chat sessions or run various models simultaneously, improving efficiency and multitasking. You can also load different models at the same time for tasks like RAG and running agents.

2. Wheebot: Create and edit landing pages within minutes using WhatsApp. Just describe your requirements in plain English, and Wheebot will generate and update your site instantly, all through secure, end-to-end encrypted chats.

3. BuildBetter: Uses AI to help product teams search recordings, generate call summaries, and update documentation efficiently. It cuts operational time to less than 30%, compared to the industry average of 73%.

4. Awesome LLM Apps: Build awesome LLM apps using RAG for interacting with data sources like GitHub, Gmail, PDFs, and YouTube videos through simple texts. These apps will let you retrieve information, engage in chat, and extract insights directly from content on these platforms.

Hot Takes 🔥

1. OpenAI blocking API access (incl. API) from China since today, new domain eu [.] api [.] openai [.] com added yesterday - sounds like the rest of the world is getting new models/API updates soon? ~
   Tibor Blaho
   

2. The gap between the people who have put in the effort to figure out how to make AI work for them and those who think it's a silly stochastic parrot is winding at a pretty rapid pace. We are still early but at a certain point it's going to be next to impossible to close that gap ~
   xjdr
   

3. the biggest drivers of ai pause sentiment won't be fears of x-risk but mass confusion as we cross the event horizon
   when sci/tech progress outpaces our sum collective understanding, the default won't be to lay back and relax imo
   even benign uncertainty is quite scary to us ~
   Heraklines

Meme of the Day 🤡

That’s all for today! See you tomorrow with more such AI-filled content.

Real-time AI Updates 🚨

⚡️ Follow me on Twitter @Saboo_Shubham for lightning-fast AI updates and never miss what’s trending!

PS: I curate this AI newsletter every day for FREE, your support is what keeps me going. If you find value in what you read, share it with your friends by clicking the share button below!

Share Unwind AI

1. China’s first real-time multimodal AI model to rival GPT-4o

2. New company is bringing Hollywood-grade AI visual platform

3. YouTube’s AI tools to remove copyrighted music without losing your sound

4. This opensource AI voice assistant responds in near real-time

& so much more!

Read time: 3 mins

Latest Developments 🌍

China’s Multimodal Model Beats GPT-4o and Claude 3.5 🌟

China’s AI progress shows that the Valley is not the only king. At the World Artificial Intelligence Conference, Chinese AI company SenseTime introduced its new multimodal AI model SenseNova 5o and the improved language model SenseNova 5.5. SenseNova 5o can process text, images, audio, and video, and is suitable for real-time interactions. The model outperforms state-of-the-art models like GPT-4o and Claude 3.5 Sonnet across most of the standard benchmarks.

Key Highlights:

1. Real-time Video Chat - Due to its vision capabilities, SenseNova 5o can recognize and describe individual objects by simply pointing a smartphone camera at the object while the AI app is running.

2. Improved Performance - SenseNova 5.5 achieves a 30% increase in performance over version 5.0, released just two months ago. It also outperforms leading AI models like GPT-4o, Claude 3.5, and Gemini 1.5 Pro.

3. Edge-based LM - SenseTime is also investing in edge-based language models. With SenseChat Lite-5.5, inference time has been reduced to 0.19 seconds, 40% faster than version 5.0, and inference speed has increased by 15% to 90.2 words per second.

4. AI Video Generator - SenseTime also demoed its video generation foundation model Vimi AI which can generate up to 1-minute clips from a single photo while providing control over facial expressions, lighting, and background.

Full Control to Creatives Over AI-Generated Scenes 🌅

The current AI video generation can be compared with AI image generation a few years earlier. It lacks the depth, details, and magic of storytelling. The technology has immense potential to overtake traditional storytelling which can’t go beyond real-world physics.

Entering this rapidly moving industry is Odyssey which is developing “Hollywood-grade visual AI,” to give filmmakers, game developers, and artists unprecedented control over AI-generated visuals. It’s an effort of a team that has previously delivered state-of-the-art AI and simulation systems at Waymo, Cruise, Meta, Nvidia, and has deep experience in computer graphics research and filmmaking.

Key Highlights:

1. Layered Generative Models - Odyssey utilizes four distinct models for geometry, materials, lighting, and motion, allowing for granular control over each element of a scene. This contrasts with single text-to-video models, which offer less flexibility.

2. Hollywood-Level Fidelity - Odyssey aims to create visually stunning and realistic results suitable for high-end productions. This is very important to counter the substandard often-unrealistic output of current tools.

3. Focus on Creative Control - A core principle for Odyssey is empowering storytellers, not replacing them. Their tools are being designed to integrate into existing production pipelines, so the users can direct and refine AI-generated content to match their vision.

Quick Bites 🤌

1. YouTube is updating its “Erase Song” feature with a new AI feature that will identify and remove copyrighted music from videos while keeping other audio elements intact. Creators can choose between “Erase Song” to target only the copyrighted audio or “Mute All Sound” to silence everything within specified timestamps.

2. The AI hardware industry is booming at rocket speed. Two weeks back Nvidia crossed $3 Trillion market cap to become the world’s most valuable company. Yesterday, Taiwan Semiconductor Manufacturing Co. (TSMC) briefly hit a $1 trillion market value. Analysts expect TSMC to raise its revenue guidance and possibly increase wafer prices due to strong market positioning. (Source)

3. This is probably the fastest AI Voice Assistant available today. Swift is an open-source voice assistant that uses Groq for fast inference of OpenAI Whisper (for transcription) and Meta Llama 3 for generating text responses.

Tools of the Trade ⚒️

1. OpenContracts: Free, opensource tool for managing and analyzing documents, especially PDFs. It extracts layout features, generates vector embeddings, and performs both manual and automated annotations to make document data searchable and actionable with querying features.

2. ChatLabs: Interact with top AI models like ChatGPT, Claude, Gemini, and Llama 3 in one place for chatting, writing, web searching, and image generation. It also offers tools for managing AI prompts, generating images, AI web browsing, using voice input, and organizing chats.

3. Open LLM Explorer: Test leading language models from the Open LLM Leaderboard on your specific prompts and view the results and inference code easily. You can try it for free

4. Awesome LLM Apps: Build awesome LLM apps using RAG for interacting with data sources like GitHub, Gmail, PDFs, and YouTube videos through simple texts. These apps will let you retrieve information, engage in chat, and extract insights directly from content on these platforms.

Hot Takes 🔥

1. In the next 3 years, we will reach AGI!
   Brace yourselves: job losses will skyrocket, and AI will become your new work partner.
   Yet, ASI isn’t here to end us…yet. The future is closer than you think! ~
   Eli Braginskiy
   

2. If intelligence is nonlinear, then the average person is pretty dumb. ~
   Naval

Meme of the Day 🤡

That’s all for today! See you tomorrow with more such AI-filled content.

Real-time AI Updates 🚨

⚡️ Follow me on Twitter @Saboo_Shubham for lightning-fast AI updates and never miss what’s trending!

PS: I curate this AI newsletter every day for FREE, your support is what keeps me going. If you find value in what you read, share it with your friends by clicking the share button below!

Share Unwind AI

1. Microsoft’s MInference speeds up LLM inference on long context by 10x

2. New Chinese opensource LLM outshines Llama 3 and Gemma 2

3. Waymo’s robotaxi gets pulled over by Phoenix police

4. AI will take center stage at Samsung’s Galaxy Unpacked event

5. Write code 2x faster with this AI copilot

& so much more!

Read time: 3 mins

Latest Developments 🌍

1M Context 10x Faster In A Single A100 🐎

LLM inference with long prompts faces a major computational bottleneck. The initial processing stage, called “pre-fill,” takes a lot of time, especially for prompts with millions of tokens. For instance, it could take 30 minutes for an 8 billion parameter LLM on a single A100 GPU to just handle the pre-filling of a 1 million token prompt.

Microsoft researchers have developed a new method MInference to tackle this bottleneck and speed up the pre-filling process. It analyzes and leverages patterns in how LLMs pay attention to text, making the whole process much faster without compromising the LLM’s accuracy.

Key Highlights:

1. Up to 10x faster - MInference makes pre-fill up to 10x faster for 1 million token prompts on a single A100 GPU. Tasks that previously took half an hour might now only take a few minutes.

2. Maintains or improves accuracy - MInference doesn’t just focus on speed; it maintains or even slightly improves the LLM's accuracy on different tasks.

3. Works with existing LLMs - You don’t need to retrain your LLM models from scratch to use MInference. It can be directly applied to existing models.

4. How it works - It first identifies recurring patterns in how LLMs pay attention to different parts of a text, then leverages them to optimize the pre-filling stage. To carry out the calculations needed for pre-fill, it uses specialized GPU kernels. This combination contributes to the speed gains.

Opensource LLM Built for Speed and Reasoning 🧠

Another Chinese opensource LLM makes it to one of the best LLMs on Open LLM Leaderboard. InternLM 2.5 is designed for speed, efficiency, and enhanced reasoning capabilities. With 7B parameters, the model boasts state-of-the-art performance on complex tasks, outperforming competitors like Llama 3-8B and Gemma 2-9B. The team has also opensourced two more models, fine-tuned for instruction following, chat experience, and function call. One of them, InternLM2.5-Chat-1M, has a context window of 1 million tokens.

Key Highlights:

1. Outstanding Reasoning - InternLM 2.5-7B-Chat performs exceptionally well across all benchmarks including MMLU, MATH, GSM8K, and GPQA. It outperforms models in its size category like Yi 1.5-9B, Llama 3-8B, and Gemma 2-9B.

2. 1M Context window - InternLM2.5-Chat-1M nearly perfects at finding needles in the haystack on 1 million long context, with leading performance on long-context tasks like LongBench.

3. Stronger tool use - InternLM2.5 supports gathering information from more than 100 web pages. InternLM2.5 has better tool utilization-related capabilities in instruction following, tool selection, and reflection.

Quick Bites 🤌

1. A Waymo self-driving vehicle in Phoenix got pulled over by local police. The robotaxi operating without passengers ran a red light and briefly occupied an oncoming traffic lane before parking. The vehicle was erratic moving preceding the traffic stop. While the police were unable to issue a citation to the driverless car, Waymo says the incident was due to unclear construction signage. (Source)

2. Samsung is heavily focusing on AI at its upcoming Galaxy Unpacked event on July 10. While current Galaxy AI features include fairly standard capabilities like on-device translation and photo editing tools, the event will be an opportunity for Samsung to unveil more unique AI functionality. Details are scarce for now, but the company is clearly aiming to make a splash in the AI space. (Source)

3. The AI industry is facing a revenue reality check. Seqioua says that despite predictions, the actual money generated by AI remains far below expectations, jumping from a $200 billion to a $600 billion discrepancy. While getting hold of GPUs is no longer a struggle and companies like OpenAI are seeing profits, the wider AI market isn’t seeing the same success.
   
   Are we in a delusion thinking that “we’re all going to get rich quick, because AGI is coming tomorrow, and we all need to stockpile the only valuable resource, which is GPUs?”

Tools of the Trade ⚒️

1. Supermaven: Fastest and free AI code completion assistant. It scans your entire codebase, making context-aware suggestions and is compatible with popular IDEs like VS Code, JetBrains, and Neovim.

2. BeyondPDF: Find specific phrases or keywords in your PDF documents instantly and locate related content even if you don’t know the exact wording. It runs entirely on your device so your data stays private.

3. Kerplunk: With this AI video interviewing tool, you can screen and interview thousands of job applicants in under three minutes, supporting 95+ languages.

4. Awesome LLM Apps: Build awesome LLM apps using RAG for interacting with data sources like GitHub, Gmail, PDFs, and YouTube videos through simple texts. These apps will let you retrieve information, engage in chat, and extract insights directly from content on these platforms.

Hot Takes 🔥

1. Sundar Pichai is the Joe Biden of AI. ~
   Pedro Domingos
   

2. Most of the internet will be AI APIs calling one another ~
   Beff – e/acc
   

3. Few-shot prompting will soon become obsolete. It is just a transitional step as we shift from machine learning to LLM-centered AI. Natural interactions will win out. ~
   Denny Zhou

Meme of the Day 🤡

LLMs when someone asks how many times the letter 'r' appears in 'strawberry'.

That’s all for today! See you tomorrow with more such AI-filled content.

Real-time AI Updates 🚨

⚡️ Follow me on Twitter @Saboo_Shubham for lightning-fast AI updates and never miss what’s trending!

PS: I curate this AI newsletter every day for FREE, your support is what keeps me going. If you find value in what you read, share it with your friends by clicking the share button below!

Share Unwind AI

Here are 10 AI breakthroughs that you can’t afford to miss 🧵👇

First Real-time Voice AI Ever Released, Moshi 👬

Kyutai, a Paris-based AI research lab, unveiled their real-time voice AI called Moshi. Developed from scratch in just 6 months by a small team of eight, Moshi distinguishes itself as the first voice-enabled AI openly available for public testing and use.

Moshi has a very realistic and emotionally nuanced voice, and can talk in 70 expressions. With a latency of ~160 ms, the conversations feel real-time and natural. Moshi’s code and model weights will be shared publicly soon.

Salesforce’s 7B Model Outperforms GPT-4 & Claude-3 Opus 💪

Salesforce has released two new language models, xLAM-7B and xLAM-1B, designed for function calling. They are particularly good at interacting with applications and services for retrieving information and completing tasks as instructed. These models stand out for their strong performance despite being smaller.

With only 7B parameters, the model achieved SOTA performance on the Berkeley Function-Calling Benchmark, outperforming multiple GPT-4 models. Even the 1B model surpassed GPT-3.5-Turbo and Claude-3 Haiku. The key lies in how they are trained.

Eleven Labs Releases Voice Isolator 🎙️

Remove unwanted background noise and extract crystal clear dialogue from any audio to make your next podcast, interview, or film sound like it was recorded in the studio. It’s not currently optimized for music vocals. Voice Isolator costs 1000 characters for every minute of audio.

Runway’s Gen-3 Alpha Model is Available for Use 🌅

Runway has made its latest Gen-3 Alpha model available publicly for use. The pricing plans to use Gen-3 Alpha start from $12 a month. Gen-1 and Gen-2 are available for free with limited credits. You can try it here.

Cloudflare Declares War on AI Content Scraping 🤖

Cloudflare has introduced a new one-click feature to block all AI bots from scraping website content, available to all customers, including those on the free tier. This simple dashboard setting addresses concerns over data ownership and the impact on content creators.

Using its extensive global network, Cloudflare proactively identifies and categorizes suspicious AI bot activity. Even bots disguising themselves as regular browsers are detected and blocked through advanced machine learning models.

Perplexity Gives Pro Search a Research Overhaul 🧑‍💻

Perplexity AI has upgraded its Pro Search to handle complex research tasks. Previously capable of quick searches, Pro Search can now execute multi-step reasoning to answer intricate questions by understanding, planning, and working through goals. This upgrade combines enhanced search and synthesis capabilities with advanced computational abilities for a comprehensive research tool.

Meta is Testing Llama-3 405B Model in Meta AI 🐑

Meta has started rolling out the Llama 3-405B model (preview) in Meta AI in the latest WhatsApp update to limited users. The screenshot shows that users will be able to select the model they want to interact with. Llama 3-70B will be the default model, and users can select the 405B model for more complex prompts with a usage cap.

Take Control of Your AI Agents with LangGraph 🎛️

LangChain has introduced stable release of LangGraph v0.1 and LangGraph Cloud in beta, an infrastructure for scalable and reliable agent deployment. These tools address the challenges of building real-world AI applications that can reliably execute complex tasks. LangGraph and LangGraph Cloud provide developers with increased control, visibility, and scalability in their agent-based applications.

Extract Data From Any Website with a Single API Call 🌐

MultiOn has released a new tool for developers working with web data: the Retrieve API. This API lets you extract structured information from any website using simple natural language commands, eliminating the need for complex parsing or scraping scripts. It integrates seamlessly with MultiOn’s existing Agent API, so you can build truly autonomous web agents that can navigate pages and scrape information in just 3 lines of code.

Microsoft GraphRAG to Teach LLMs Beyond Keywords 📚

Traditional RAG is a tough nut to crack. It struggles to connect related facts scattered across a dataset and often fails to grasp the bigger picture within a large corpus of information. This is where Microsoft Research’s GraphRAG steps in.

Unlike traditional RAG, which relies on simple keyword matching, GraphRAG builds a knowledge graph from your text data, so the LLM can understand relationships and meaning in a way that’s not possible with keyword-based searches.

Which of the above AI development you are most excited about and why?
Tell us in the comments below ⬇️

That’s all for today 👋

Stay tuned for another week of innovation and discovery as AI continues to evolve at a staggering pace. Don’t miss out on the developments – join us next week for more insights into the AI revolution!

Click on the subscribe button and be part of the future, today!

Subscribe now

📣 Spread the Word: Think your friends and colleagues should be in the know? Click the ‘Share’ button and let them join this exciting adventure into the world of AI. Sharing knowledge is the first step towards innovation!

Share

🔗 Stay Connected: Follow us for AI updates, sneak peeks, and more. Your journey into the future of AI starts here!

Shubham Saboo - Twitter | LinkedIn ⎸ Unwind AI - Twitter | LinkedIn

Awesome LLM Apps | Sponsor Us