Runway Gen-3 Takes Off, OpenAI Sora Remains Grounded
PLUS: Knowledge Graph for RAG, Codestral in Jupyter Notebooks, Opensource desktop app to run local LLMs
Today’s top AI Highlights:
Why AI-generated text feels repetitive
Microsoft GraphRAG helps LLMs connect dots
Runway’s Gen-3 Alpha model is available for use
Figma pauses its AI design feature after accusations of plagiarism
Use Mistral AI’s Codestral in Jupyter Lab with Jupyter AI
& so much more!
Read time: 3 mins
Latest Developments 🌍
LLMs Use Syntactic Templates From Training Data 📝
Certain sequences of words make it certain that a text is AI-generated. Think about “delving into it” or “revolutionizing how…” or “in a significant advancement...” Ever thought about how LLMs like GPT-4 or Gemini get their writing styles? A new study explored how LLMs learn and utilize syntactic patterns from their training data.
This study introduces the concept of “syntactic templates” – like blueprints for sentences – to measure repetition in generated text. By analyzing various LLMs, researchers found they rely on these templates far more than human writers do.
Key Highlights:
LLMs stick to familiar sentence structures - The study reveals that 76% of the grammatical patterns found in LLM-generated text come directly from their training data. This is much higher than the 35% observed in human-written text. It suggests that LLMs might be over-reliant on what they’ve memorized.
Sentence patterns can reveal training secrets - Even for closed-source LLMs, analyzing their preferred syntactic templates can offer valuable clues about the data they were trained on.
Spotting sneaky memorization - The study proposes using these templates to detect a more subtle form of memorization, where LLMs may not reproduce exact phrases but still rely heavily on memorized sentence structures. This “soft memorization” was found to be surprisingly common.
GraphRAG to Teach LLMs Beyond Keywords 📚
Traditional RAG is a tough nut to crack. It struggles to connect related facts scattered across a dataset and often fails to grasp the bigger picture within a large corpus of information. This is where Microsoft Research’s GraphRAG steps in. Unlike traditional RAG, which relies on simple keyword matching, GraphRAG builds a knowledge graph from your text data, so the LLM can understand relationships and meaning in a way that’s not possible with keyword-based searches.
Key Highlights:
Uncover Hidden Connections - GraphRAG connects related information within a dataset, even when it’s spread across multiple documents. This helps LLM to tackle complex questions that require understanding the relationships between different entities and events.
More Accurate Results - Instead of keyword matches, GraphRAG uses its knowledge graph to pinpoint answers that are both accurate and contextually relevant. And for added trust and transparency, each part of an answer can be traced back to the original source material.
Extract Meaningful Insights - GraphRAG goes beyond retrieving individual facts. It can automatically identify overarching themes, pinpoint key trends, and offer insights that would be extremely difficult to unearth using traditional methods.
Quick Bites 🤌
Last week Eleven Labs launched its Reader App that can take articles, PDFs, ePub, newsletters, e-books or any other text on your phone and turn it into an emotionally rich, context-aware voiceover. They are bringing the voices of iconic stars like Judy Garland, James Dean, Burt Reynolds, and Sir Laurence Olivier to the Reader App so you can listen to your text in these legendary voices on-the-go.
Runway has made its latest Gen-3 Alpha model available publicly for use. The pricing plans to use Gen-3 Alpha start from $12 a month. Gen-1 and Gen-2 are available for free with limited credits. You can try it here.
Figma has temporarily disabled its AI design feature “Make Design” after the company was accused of plagiarizing Apple’s Weather app. Andy Allen, founder of NotBoring Software, first raised the alarm on X when he noticed the tool consistently generating designs resembling Apple’s app.
😍 Enjoying so far, share it with your friends!
Tools of the Trade ⚒️
Codestral in JupyterLab: Jupyter AI now seamlessly integrates Mistral AI’s Codestral into JupyterLab for a streamlined and enhanced AI-assisted coding experience within the Jupyter ecosystem. Install needed packages in your Python environment, and launch JupyterLab!
pip install jupyterlab langchain-mistralai jupyter-ai pandas matplotlibGPT4All 3.0 by Nomic: An open-source, private desktop app that helps you run thousands of models locally across all major operating systems. It features major UI/UX improvements and a Local Document Chat powered by Nomic Embed for a user-friendly and private experience.
EverArt: Train AI models on your brand’s products, styles, or mood boards to generate high-quality images and branding content. You can create media at scale, fine-tune models with ease, and collaborate with your team.
Awesome LLM Apps: Build awesome LLM apps using RAG for interacting with data sources like GitHub, Gmail, PDFs, and YouTube videos through simple texts. These apps will let you retrieve information, engage in chat, and extract insights directly from content on these platforms.
Hot Takes 🔥
Met an OpenAI engineer that so strongly believes we’ll have ASI soon that he sold his SF house and is looking to short the housing market of any cities that rely on human intellectual capital ~
Jordan BurgessReality: OpenAI has already won and become a black budget project.
GPT6 gov supercomputer probably being built rn ~
Beff – e/acc
Meme of the Day 🤡
That’s all for today! See you tomorrow with more such AI-filled content.
Real-time AI Updates 🚨
⚡️ Follow me on Twitter @Saboo_Shubham for lightning-fast AI updates and never miss what’s trending!
PS: I curate this AI newsletter every day for FREE, your support is what keeps me going. If you find value in what you read, share it with your friends by clicking the share button below!