Daniel Tunkelang

We are not slowing down on Gen AI, come see why...and how it changes things, the nature of work. https://lnkd.in/gW3ettg3

2

I just read an insightful piece by Eric Broda (published in Towards Data Science) that outlines the Agentic Mesh—a new ecosystem designed for autonomous agents to interact, collaborate, and transact. His work introduces a structured approach for managing the coming surge in autonomous agents, showing how they can function independently yet collectively. Agentic Mesh serves as a networked environment where autonomous agents operate in a safe, transparent, and efficient manner. It allows agents to find each other and securely interact under a framework of policies. Each agent has a specific purpose and autonomy, making it a valuable contributor to the ecosystem without sacrificing security or control. Core Components: 1. Marketplace: A centralized space where users discover agents by tasks and capabilities, offering a trusted environment for engagement. 2. Registry: A metadata repository organizing each agent’s purpose, owner, and policies, allowing agents and users to securely find and connect with each other. 3. DNS Integration: Each agent has a recognizable digital identity, enabling seamless discovery and interaction, just like an internet service. 4. Agent Stack: This layered structure of communication protocols, control services, and interfaces helps agents operate effectively and responsibly. It ensures that agents act independently within controlled parameters. Agentic Mesh’s effectiveness depends on agents sharing consistent attributes, including: • Purpose: A clear objective drives each agent’s actions. • Ownership: Transparency around ownership fosters responsibility and trust. • Autonomy: Agents act independently within set parameters to complete tasks. • Discoverability: Easily locatable, ensuring efficient interaction within the network. • Intelligence: Agents utilize language models to plan and execute tasks, adapting to complexity as needed. Operational Flow: Registration, Discovery, and Execution Agent interactions in the Mesh follow three core flows: 1. Registration: Agents set up purpose, policies, and registration data to gain visibility in the mesh. 2. Discovery: Through the registry, agents and users locate potential collaborators based on defined needs. 3. Execution: Engaged agents use AI models to plan and perform tasks, enlisting collaborators when necessary to complete workflows. Read the full article: https://lnkd.in/eVmq5BFC

16

1 Comment

A really practical guide to LLMs is now available! Learn what to do, what to avoid, from prompting to RAG, to fine-tuning, and evaluation. Definitively worth reading. https://lnkd.in/dCC6uKjs

34

#RAG DOES NOT REDUCE HALLUCINATIONS IN #LLMS: Think I've called this out a few times, but the validation is nice - https://lnkd.in/eAi6zxW2. The article makes two well established points: 1) "Hallucination is a problem of reasoning and not relevance! Any amount of relevant text fed through RAG to a machine will retain the original perplexity and entropy in the system to hallucinate independent of the text"; and 2) "The initial challenge arises from the retrieval phase’s reliance on semantic similarity, which is quantified by inner product spaces or cosine similarities. This metric, while capturing semantic closeness, does not differentiate between factually accurate and inaccurate information." This should be clear to everyone, but the industry still shills flimsy RAG scaffolding anyway (shovels to gold miners).

104

32 Comments

🚨 LLM Myth-Busting Alert! 🚨 For those who don’t know, Large Language Models (LLMs) have difficulty learning new data. Most of what they know is already months old and only gets updated with new releases. The options? Either you spend a LOT of time, resources, and datasets #finetuning (good luck with that) or rely on #RAG (like having a separate database that the LLM taps into). But here’s the kicker: at Attentio, we can insert new facts into an LLM in seconds with as little as ONE data point. Yeah, we said it. Don’t believe us? Check it out for yourself in this video 👇 #AI #LLM #innovation #tech https://lnkd.in/g45GmpbZ

8

🆕📨 Newsletter 🚀 Vector Search Unpacked (with Prashanth Rao of Kùzu Inc.): - Comprehensive guide on choosing the right vector search system - Importance of scalability, including horizontal scalability and real-time index updates - Prioritize performance features like real-time index updates, robust vector index support, and hybrid search - Consider deployment options, scalability requirements, and integration with existing infrastructure https://lnkd.in/gchQKFU3

25

An off-the-shelf LLM that hasn’t been grounded in your data can possibly generate inaccurate results and even hallucinate. Enter prompt grounding — the act of adding your data and metadata to the LLM for better, accurate results. Here's how it works:

21

Small Language Models (SLM) are being discussed a lot lately, but why? Well for starters LLM's are costly and slow. SLMs are the fields newest solution to the bank account draining capabilities of LLMs, but most of the writings I've seen on SLMs is focused on the technology and use of the SLM itself. The real power of smaller models (even traditional MaxEnt and CRF models) is that they are incredibly fast and practically free in comparison to an LLM. What's been missing in the SLM discussions that I've seen is the idea of a hybrid approach where you use LLM's to feed high quality training data to SLMs so that you have amazing insights that can run on high volume content streams at a very low cost. Something to think about...

31

Today's article on MCMC - read to learn why there are better and more efficient techniques than chasing LLMs and other #AI buzz. https://lnkd.in/eaw6Paan

8

Checkout my recent Forbes article on counting the cost of an AI deployment.

22

1 Comment

This week I went to the first user group of Cube, thanks to Artyom Keydunov. It was a great event with many new cool announcements from Cube that I'll separately post about. After half a day spent talking with users, and experts such as Josh Klahr (Snowflake), Francesco Puppini (Prof and Author of The Unified Star Schema), Katie Hindson and Hamzah Chaudhary (founders of LightDash), Galen Marchetti etc, and reading more after the event from the best minds like Sanjeev Mohan and Professor Christopher Manning, a question popped in my mind: Are Semantic Layers the treasure map for LLMs? Semantic layers are not new, they started in 1991, when BusinessObjects patented the semantic layer. It’s a great concept but it hasn’t really taken hold in the enterprise. Now LLMs are generating renewed interest in the semantic layer. Semantic layers really unlock the future of where things will go with AI. If you want trustworthy data, you need a semantic layer. If you let an LLM loose on a schema versus let an LLM loose on a semantic layer on top of that schema with some basic curation, you're going to get much better results. My hot takes from the blog post below: 1️⃣ Text-to-SQL startups, while hyped up, are destined to fail without a semantic layer.  2️⃣ Data Products are trending: It's becoming expected in your product to feed some data back to the person paying for it. The intersection of semantic models, plus data sharing and distribution is extremely powerful.  3️⃣ To unlock LLMs, we need first to write the literature of data modeling for data analytics. This will serve as the foundation of the dialogue that we will then have with LLMs.  4️⃣ The industry has been talking about a universal source of truth for semantic layers but it's not going to happen, because it's not how reality works. 5️⃣ We’re entering a golden age of vertical specific analytical software to be built on top of semantic layers. The treasure that semantic layers help uncover isn't just gold and jewels, but something far more valuable in the modern business world: trustworthy, actionable insights. By providing a well-defined framework for understanding data relationships and business logic, semantic layers enable LLMs to dig deeper and extract nuggets of wisdom that might otherwise remain buried. Read more below 👇

39

11 Comments

Gemini 2.0 handles complex math and planning tasks previously out of reach of LLMs. You can turn on "code generation" in AI Studio and in the APIs. For context, let's talk about function calling and voice AI agents. (I spend a lot of time working on voice AI.) Voice AI agents use function calls for: - Interacting with external systems specific to particular use cases and deployment contexts. - A wide range of RAG-like information lookup tasks. - Conversation persistence, including complex actions like saving a privacy-redacted version of a conversation. - Script following. Some of these uses were obvious before we had built and deployed production voice AI systems. But some of them weren't. (Script following, for example.) We had to build experiments, and then production systems, to figure out what features and tooling these new systems really need. I'm convinced that code generation will be as important as function calling, once we've accumulated some experience building with Gemini 2.0 and similarly capable new LLM versions. Credit for this insight goes entirely to Shrestha Basu Mallick, who shepherds work at Google on the Gemini APIs and AI Studio. When Shrestha first told me that really good code generation was going to make a lot of new things possible, I didn't really get it. But after playing with code generation for a few weeks ... I still don't fully get it, but in a good way, now. It's clear that there are some obvious things to explore: - Context-aware math on demand driven by natural language. - Generative UI. - Better "binding" behavior with function calling. But I'm also convinced now that there is a lot of unexplored, non-obvious (at least to me, yet) territory. Let's take that last one — improved function calling performance. Function calling and code generation go together like peanut butter and jelly. Functions often take arguments that are fairly complicated. With code generation enabled, Gemini has access to this new pattern: natural language ➡️ code that creates argument structs for functions ➡️ multiple function calls Think about how powerful this is. The ability to use even very basic code structures — loops, for example — massively expands what LLMs can do. In fact, my impression is that function calling in Gemini 2.0 is now trained as a subset of the model's code generation capabilities. This feels quite different than how function calling works in other models that I have access to. In other models, function calling feels like a close cousin to structured output. Which is great. But this new "modality" is even better. (More on the prompt in this post, and the meta-prompt for this post, here: https://lnkd.in/gycKtnNX)

49

9 Comments

inception llms: experimenting with a new LLM design pattern where I prompt Gemini to write code on my behalf which, in turn, uses Gemini to perform tasks - So meta, isn't it? One prompt for one task. in less than two hours created a full pipeline which would have taken days otherwise. Gemini wrote more than 97% of my code, discoverying libraries I was not aware of - overall, saving me tons of time as swe. Where do you think you can use this power in your product? try it yourself next time you develop a feature. I named it "inception llms"

76

10 Comments

My book, Visualizing with Text, came out just before LLMs. Is it applicable? I outline a half dozen application areas where LLMs + Viz will be transformative and provide more than a dozen examples from recent work, all leveraging approaches from the book: https://lnkd.in/gS2fMhpe

50

3 Comments

At a Generative AI thought leadership dinner with other CTOs in NYC today and some interesting trends are emerging: * a consensus that LLMs are converging on a general set of features and basically comparable performance * a consensus that an org's data pipelines will matter more than the LLM chosen * and a group of people who read the TOS that are really scares about what these LLMs will do with access to all your data. Lastly, my prediction: the work of tech is moving from the keyboard to the whiteboard. Ai as microservice means structuring your data, thinking with ai, and chaining ai agents will supercharge your career and your cause -- but only if social change orgs platform their data. In my study of polymaths, Charles Babbage designed the basic idea for a unit of computing being something that takes an input, does something to it and gives an output -- the so called Babbage difference engine. In a real way, GenAI is approaching a world where chaining inputs ala a babbage difference engine is accessible to all. This can be an awesome opportunity if we get Non profits to work together!

11

3 Comments