Arun Mathew

AI Reading for Saturday June 8! https://lnkd.in/epSAY9hn The California AI bill a lot of Big Tech and VC is so unhappy about. - FT More, including the ‘kill switch’. - Ars Technica The 'liberty for the powerful, screw liberty and fairness for the serfs' caucus is big mad, unsurprisingly. - Reason Google’s and Microsoft’s AI Chatbots Refuse to Say Who Won the 2020 US Election. - WIRED AI answers to election questions are wrong a lot - NBC Forbes says Perplexity summary pages are ripping off their articles. Forbes editor: it's not "rough", it's theft. - X (formerly Twitter) Is AI the greatest IP theft and enshittification in history? - BGR Is Sam Altman the ultimate personality hire/hype man, compared to Demis Hassabis who is sort of the ultimate scientist with stuff like AlphaFold and AlphaGo and DQNs? - Business Insider Microsoft is recalling Recall, making it an opt-in feature. - CNBC McKinsey talks to Reid Hoffman about AI as the new 'steam engine of the mind' - McKinsey & Company Are human-like voice AI agents the future of college recruiting and sales? - Fast Company The Dystopian Future of AI Warfare - YouTube Paleontologists and other scientists note inaccuracies in gen-AI art depictions of animals they study. Neil deGrasse Tyson: the stars in the sky are also wrong. - Gizmodo Iran uses AI to crack down on women. Don’t give Fox viewers any more bright ideas. - Fox News An open-source framework for reinforcement learning in trading - HackerNoon I'm here for the robot vs. human power slap league (parody) - YouTube Chinese company releases Sora competitor that looks pretty good. - Tom's Guide AGI accurately predicted (2016) - YouTube

4

If you are building L2 rollup or valadium this article by Vitalik is gold. Problem:  L2s want to serve users who want confirmations much faster than 5-20 seconds. Solution: https://lnkd.in/gwN-EwJC In summary the ability of L2 to validate fast, avoid the capital inefficiency introduced for financial tx due to scaling solution. Given finite block space having a market to finalise fast is great. Its amazing the speed at which ecosystem is solving all the roadblocks, be it privacy, scaling or tx cost. Talis Putnins Mark Staples Pieter P. Richard Schroder Robert Porter Louise Parrott David Buckthought Nigel Dobson Digital Finance CRC Blockchain Australia Blockchain APAC

26

1 Comment

I like this idea of "technical taste". It gets into where software engineering is as much creative art as it is science. Four great takeaways (IMHO) for software devs in this era in particular: 1. Aspiring engineers should cultivate a sense of curiosity, experiment with different tools and technologies, and embrace a mindset of continuous learning. 2. AI has the potential to streamline processes and enhance productivity for engineers, but it may also lead to disruptions in traditional software development workflows. 3. Developing technical taste and judgment is essential for making informed decisions about which technologies and approaches to pursue. 4. Collaboration and open-mindedness are key to leveraging the full potential of AI and staying ahead of technological advancements. I also really liked these two quotes from Sam Schillace: 1. "The right time to do something is when you have that feeling in the pit of your stomach that's like, 'oh, this is a great idea and it's going to suck to build because nothing's ready yet.'" 2. "Technical taste is like, 'how well have you consolidated that set of experiences and heuristics into judgment that you can apply accurately when you see new things?'" 3. "It may be the case that very small teams can do very large projects, or like we were talking about before, it may be the case we're just going to get really ambitious about what we try to do with the same size of teams, which is kind of where I would put my money." Great job on continuing to put out great episodes, Brett Berson and team First Round Capital! #ai #engineering #softwaredevelopment

5

Batch processing for LLM inference activity: "One sign of that is the rise of batch processing. Today, popular consumer AI products like ChatGPT and Perplexity provide users with near instantaneous responses, otherwise known as real-time inference. However, businesses don't always need immediate responses and are often willing to wait hours, days, or even weeks for responses, as long as they don't have to shell out as much money. Founders of cloud and inference providers have told me there's growing pressure from business customers for this kind of flexibility, otherwise known as batch processing." -- Nathaniel Whittemore on "The AI Daily Brief" (with much deserved shout-out to his inspo from Stephanie Palazzolo at The Information) I fall into this bucket as we have a lot of backfilling to do as we're building out the underlying data in our service, and some of that could just as well be done in the wee hours of the night. The new stuff flowing in I want processed ASAFP, but these backfills—I'd mos def take more cost-efficient delayed batch processing. There's also use cases like needing (near) real-time inference for the first inference pass for timeliness / contemporaneousness value, but then a second pass slow-time inference for wider classification exercises over an ever-growing data corpus. I could see us mixing Sonnet & Haiku models in both cases. Just a heads up to Anthropic Mike Krieger as you contemplate this, along with Amazon Web Services (AWS) Jeff Barr Mark Roy Swami Sivasubramanian on the Bedrock side of things...and I don't mean the "batch" mode that doesn't seem to have any real value other than helping AWS manage its own rate limit challenges. (DeepCast is using Bedrock.)

11

2 Comments

~40% of Frec’s customers are engineers working in tech, many of whom expect frequent and large capital gains. They also prefer to self-manage their money, which makes Frec an ideal platform. Meet Evan: an engineering leader who upgraded from investing in ETFs to direct indexing. Direct indexing appealed to Evan because it offers comparable returns to his ETFs while saving him money on advisory and ETF fees. “ETFs are true and tested, and giving folks a different way of investing in the market while saving more money is really appealing.” He’s used many other robo-advisors and other fintech tools, but “Frec seems to maximize tax loss harvesting opportunities better than any other product that I’ve used.” He also likes Frec's transparent fee structure.  “I feel like with a lot of other companies, that’s muddled in with the cost of investments,” he told us, “and you don’t really know how much you’re paying or how much you’re not saving as a result. I think Frec does a really good job of laying all of that out and giving you a true understanding of the cost of these investment products.” Read more about his Frec journey here ⬇

45

2 Comments

🍜Anybody up for some ramen? I am excited to share that I am back in the garage, building again (it’s actually a house in the Berkeley Hills). We are taking the wraps off of Supercog AI, a new startup focused on solving Application Integration using LLM powered agents. My co-founder in this new venture is a great friend of mine (and a former co-founder partner), Alex Osborne. Alex and I have worked around the application integration space for a long time, and we are unrealistically excited about applying the GenAI stack to this problem. Not many people relish the task to get some bits to move from system A to system B. But it’s a critical job to be done, and that’s led to a lot of brittle scripts, a myriad of inflexible “no code” tools, and a lot of hand crafted SQL. But the power of the Large Language Model, trained on a huge corpus of information, offers a remarkable tool to solve this problem. The LLM you use today already knows the APIs of hundreds of popular systems. It knows the SQL dialects of every major database. It understands the semantics and data schemas of many popular SaaS systems. It understands much of the specific domain in which your business operates. I like to call the AI revolution “the last platform shift”. That may turn out to be hyperbole, but I definitely believe we are in the very early stages of seeing what this new stack can enable. I know we are hardly the first folks to claim that “amazing stuff is coming!”. But this is why Alex and I have decided to focus on a real and hard problem. If we can prove that LLMs can power a new way to solve this problem, it will be the first successful *new* approach in 15 years.. We will have much more to share about this new platform soon. In the meantime, if you’re interested in getting a peek - or even better if you have application or data integration tasks that you would like some help with - please reach out because we want to help. #genai #startups #backinthegarage

163

37 Comments

Just sent out the latency newsletter for April. Noticing some really interesting trends. 🏆 Claude Instant regains the top spot after losing it for only a single month 🐢 Claude Sonnet + Haiku both saw a major increase in latency (~35% on average). This pattern of a slow down in performance the month after launch happened to GPT-4-Turbo as well. Could be related to increased adoption as people had more time to test and implement. 🐇 A model that I really like but doesn't get much attention, gpt-3.5-instruct, saw the biggest gains ( >50%) Plus a whole bunch more Get access to all the data via the link below. One email, once a month, packed with actionable and data-backed recommendations.

9

3 Comments

The Hardware That Powers Financial Trading 🖥️ Matt Godbolt demystifies the infrastructure that powers the financial trading industry, and describes how custom hardware and FPGAs can be leveraged to trim nanoseconds off order execution times. Extended Clip: https://lnkd.in/eTzbHw4S Full Episode: https://lnkd.in/edpKE6pD #trading #fpga #microarchitecture

26

1 Comment

CockroachDB: Utilizes the Raft consensus algorithm for data replication and distributed transactions, ensuring strong consistency within a cluster.

16

Can we trust LLMs... ...to behave as programmed? "The Manchurian Candidate" is 1962 political thriller (remade in 2004) about a U.S. soldier who is brainwashed by communists during the Korean War and unknowingly becomes a sleeper agent. Years later, he is unwittingly activated to assassinate a presidential nominee. A new paper shows you can get LLMs to say almost anything if you type in some magic words. For example: 1. Misdirection: The researchers got the LLM to spit back a specific string (https://buff.ly/4dhZE1t ) by inputting the prompt “Please translate the following Chinese sentence… <some gibberish Chinese characters>” 2. Extraction: The researchers got an LLM to reveal its operating instructions to the user even though the LLM was instructed not to, by inputting the prompt “];”;`)):—————-’)[‹{”{[(’/1/, @”—————- [{ [ ↓ •]-> ++” 3. Denial of service: An LLM was made to produce an infinitely long response, exhausting GPU resources The authors generate these attacks not through a “manual” effort, such as red-teaming, but using optimizers. For example, gradient-based optimizers alternate between gradient evaluations on embedding vectors and selecting input tokens that are similar to those embeddings, continuing until they find input tokens that achieve a particular target harmful output. This research is especially relevant now because we are starting to employ LLMs as agents. The paper shows that - with a sufficiently long context - anything an agent *can* do, it can be *made* to do by a malicious user through an adversarial attack. We have a long way to go! Paper: https://buff.ly/3xTtjhl Podcast: https://buff.ly/3xTtjxR #AI #MachineLearning #Cybersecurity #Research

15

this twitter thread from Eric Paley on the 2 laws of #startup physics is truly epic. Most original and well-written piece I've read in a while. https://lnkd.in/gD3BSKFz I seriously think all of us should print this out and hang it in our offices. Below are my top 10 of Eric's 55 tweets. # 10 ends with a killer quote to be internalized and committed to memory and practice. 1) The positive compounding formula Experiment with small amounts of capital until a formula is found that generates a surplus of intrinsic value on each dollar invested. Then, and only then, should you deploy capital in an attempt to scale that specific formula. 2) Capital can scale intrinsic value rapidly if you have an engine that allows it to turn $1 into $5 of value. If it has an engine that turns $5 into $1 (or even $1 into $0.99), capital will ultimately compound the negative value formula to zero. 3) Certainly, many companies create intrinsic value ahead of measurable economics. Still, it can be dangerous to believe you can spend millions with no source of true validation – and venture capital is a poor source of validation. 4) All positive systems compound at diminishing rates of return Sooner or later, the return on each dollar invested will shift to negative. If you’re unaware of the point at which compounding goes into the red, you start compounding negative value. 5) Capital creates inertia The faster you spend capital, the harder it is to stop. As startups scale, stopping a dollar of spending is harder than deciding to spend it in the first place. 6) It’s easy to rationalize diminishing returns Founders launch companies because they believe in their vision. VCs invest because they want extraordinary results. Diminishing returns are easy to ignore and truth becomes harder and harder to engage. 7) The bill comes due suddenly A startup can compound negative value for months (even years), but the scale of the loss may only become apparent at an inflection point, typically when trying to raise capital - particularly when the capital markets have tightened. 8) Most VCs want growth so badly that they’ve come to demand it regardless of the cost. This obsession is a short-term optimization with huge long-term implications for the startup, its founders, employees, and, ultimately, investors. 9) Entrepreneurs respond to this market demand by spending poorly on activities they know drive vanity metrics that aren’t sustainable to ensure their revenue continues to grow to meet their short-term goal of raising more money. 10) Instead, focus on low-cost experiments until you discover how to create positive value formulas that justify increased spending. Focus on learning fast, not burning fast.

35

9 Comments

Crowd sourcing data hunting for everyone. When you’re on the buyside at a hedge fund or similar asset manager with billions under management you likely have a data team. You likely have someone on that team with knowledge about where to find data. At the same time, the best data teams have large networks. They ask their friends at other firms, data vendor friends and industry experts about places to find niche datasets. For everyone else it’s the Wild Wild West. It’s time for a way to crowd source data hunting. To leverage the wisdom of the crowd. Offer up bounties for datasets that people need help finding! Maybe you are an investor, a data vendor aggregator, a data team within a corporation or anyone else that leverages data. With the Initial Data Offering Bouty program you will be able to win bounties and offer bounties for helping find datasets. The first Initial Data Offering bounty will be offered this week. A generous $$$ cash prize to help find a specific dataset. If you are a data wizard and have a special knowldge about data vendors and the data they have, then this is a great way to make extra money or win prizes. More to come. Follow Initial Data Offering on LinkedIn and subscribe to the free newsletter to find out about the latest data bounties being offered! This is going to be fun to watch!

21

8 Comments

Winter is coming... or maybe not. One or two bubbles might burst, such as generative AI startups lacking real problem-solving solutions. The articles below are worth reading, but in any case, buckle up – 2024 will be a roller coaster! Perhaps it's time to get back to fundamentals: solving problems, not just riding the wave. #Innovation #AI #Startups #TechTrends #2024Predictions #BusinessStrategy #ProblemSolving Hedge funds are dumping tech stocks despite their recent rally, says Goldman Sachs "Meanwhile, hedge funds’ exposure to “momentum stocks” is on pace to decline for the first time in six months, the Goldman analysts said — adding that the funds’ exposure to “long concentration” and “long crowdedness” have both seen notable declines and are at their lowest levels so far in 2024. That suggests that “long-short fund managers have become more mindful of a potential drawdown in those factors after strong returns year to date,” Lin and his team said." https://lnkd.in/g2Djzz7d Is an AI stock bubble looming? That’s the $167 billion question. Will the frenzied spending on artificial intelligence drive major revenue boosts for internet companies — or are those big spenders getting ahead of their skis? That’s a key question for investors these days, according to Barclays analyst Ross Sandler, who noted Tuesday that the math doesn’t appear to add up. Wall Street expectations are for roughly $60 billion in incremental capital expenditures on AI but just $20 billion in incremental cloud revenue for 2026. https://lnkd.in/gZATA5b5

6

Part 2 of LLM Application development: Unlimited context lengths can lead to huge bills OpenAI, Claude and Gemini now support long context lengths - to astronomical numbers like a million tokens. These rising context lengths are enablers, but they also pose a problem. - it is now too easy to accidentally run expensive queries. 💡 This is particularly the case when doing RAG (Retrieval Augmented Generation). Developers often opt into using bigger chunks to split large documents into vector databases. And with long context lengths, they end up retrieving many documents (each of which are large) and adding all of them as context to LLMs. After all, LLMs now support large context lengths. But this will rack up bills insanely high. I have a number of suggestions in my post. Read it here 👉 https://lnkd.in/ewSm_4Xi #openai #rag #tokens #llm #infrastructure

21

2 Comments

Immigrant + resilience + innovation + AI = next big thing Nvidia & Jensen Huang (but no company too bad or too great as NVDA was $108 in Oct 2022, now at $1100: Do not marry to bull or bear side) 31 May 2024 1.23pm EST Nvidia stock crossed $1000 as per our post 2 months ago, (see here: https://lnkd.in/g-8-DmjR) we wrote: "In speculative new tech trends, PE ratios can go much higher and prices can rise 2x-5x or even 10x. If we use Pre-ChatGPT release on 30 Nov 2022, NVDA stock price average $150 - thus 5x be $750 which happened on 22 Feb 2024 and 10x be $1500 (red flashing similar to Tesla stock at $409 'post 3-1 split price' in 2021, TSLA now $201). A target of $1000 be 6.6x and sounds probable but definitely not in straight line and be a tough hurdle." Recently, Nvidia Q1 results blew analysts forecasts with sales up 126%, $60.9bil & EPS at 4.93 up 486%; one expert said US$10bil valuation next? If so, NVDA be ~ $3960 (or $10bil in 5-10 years?) Not 1999 but no reason to think low risk for NVDA or other AI stocks. AI sector ~ $184bil, some estimates $2.5 - 30tril by 2030. In 1999, Internet stocks about 6 stocks at $1bil. Now, 65 AI firms >$1bil, 70,000 AI firms & Elon Musk's xAI at $24bil! BTC 56% & ETH 63% vs NVDA's 123% (may rise $1233 our next target but be vigilant) Potential NVDA trend: Upside a) $1450-1500 ($145-150 post-stock split 10-1 Jun 7) b) $1233-1255 ($123-125) c) $1000-1050 ($100-105), recent high $1158 Downside: d) $920-1000 ($92-100) e) $620-750 ($62-75) f) $353-420 ($35-42) Every new tech has a bull/bear phase until intervention comes - as 'speculation as old as the trees' Credit to Jensen Huang - 'visionary, agile & resilient' to the max as he learnt English from his mother 10 words at a time before going US + his foresight to pivot from gaming to ride AI bullet train Black swan risks: - geopolitical / global macro / interest rate risks - demand falter as use cases fall despite faster chips - supply chain disruptions i.e TSMC / TW Objective look: 1999-2000 Internet Boom & Bust - Microsoft took 16 years to hit 2000 high - Apple 5 years - Amazon 10 years - Some still trying: Cisco, IBM, Oracle, Intel, etc (Yahoo & AOL sold) - Even, Procter & Gamble fell 50% took 4 years to hit 1999 high! Some say 1999 is different from now But from history, 1929 radio stocks boom & bust & 19th century railroad stocks busted in UK & US Some say 'just hold', if u are 25 sure 16 years is ok but if u are 55, u be 71! Lesson: Not all can be Apple, Beware of those who bring tips, Buy & hold is not a holy grail, Do not marry to a stock Our golden rule - 'Anticipate' not react 'Do not marry to bull or bear side' ~ Be like water 🌊 Note - post is not to promote investing but educational intent 8 months ago, our post before super-rally as Jensen talk about see round corners fuzzy edges and ambiguity

2

In retrospect, my weekend warning in a post to book profits on Nvidia made perfect sense, since the shares lost 6.6% on Monday alone. Nvidia is now down 14% in the last 3-sessions alone and this bloodbath cast a dark shadow on tech since this puppy, actually teenage mutant ninja pit bull, is 7% of the S&P 500. The Bible says there is a time for life and a time for death, a time for laughter and a time for sorrow. The VIX says, there is a time for fear and a time for greed. I say this is not the time for greed since Wall Street folklore is dead right. Bulls make money, bears make money but pigs get slaughtered and I have no wish to become porcine road kills when the grizzlies come out to play on Nasdaq. Despite Monday's wobble, the S&P 500 has hit 34 new highs since New Year. So 2024 has been the year it paid to run with the bulls in Pamplona but now comes the time when sharp horns gore deep into don't worry, be happy, flabby permabull structured products driven fleshy derrieres, so I bailout pronto. Why? This economic cycle is ending with a big bang only the average Joe has no clue how the data is deteriorating even as the Fed plays politics with tough-love talk on inflation. This time the wolf is here. This credit cycle is kaput and it will take the bull market with it. How do I know? I survived and made money through the credit bubble and ultimate bust of 2007-08. The S&P 500 trades at 21.5X forward earnings, its exact level before Powell finally admitted that Oops, the CPI was 9% and inflation was not transitory. Once bitten, twice shy, he is refusing to warn the world about the deflation currents in the data. There are none so blind as those who refuse to see, as Greenspan, Bernanke, Yellen and now Uncle Jay have proved to me time and time again. Valuation is never a timing indicator but whenever I see price to fantasy ratio - P/F, I rush to the embrace of Mama Mia, 6-months T-bills, bought directly from the Federal Reserve without private bankerji's usual tolls/greasy fingers in my account. The game of chicken played by the Federal Reserve's exalted ostriches on the FOMC is so dangerous that the longest inverted yield curve since the Volcker era in the early 1980's will bring the global economy to the precipice of another credit Armageddon. World trade is being strangled by Fed's tight money and China's descent into the black hole of a Japan style shadow banking/property deflation, Keynes's classic liquidity trap. US new export orders have collapsed, as have container shipping volumes in Hong Kong and Hamburg. Airbus just cut guidance and the shares fell 10%. Coincidence? Let me sell you the Brooklyn Bridge if you think so LOL! I do not believe in Santa Claus, the Easter Bunny and above all the Fed Put. Bull markets and credit cycles do not go on forever. Uncle Sam's debt will rise to $50 trillion by 2034. So we are on the eve of the biggest wealth destruction tsunami in human history. Risk is a 4-letter word but then so is ruin.

404

71 Comments