[Weekly] What’s your unpopular opinion about AI?
What do you believe about AI that most people here don’t, and why do you believe it?
One thoughtful comment unlocks the ability to create threads.
105 comments
Sort of agree! For me, Gemini has the best writing and conversational style of them all. But Claude is my daily coding driver, even though I don't enjoy its tone and really can't stand how it writes! Never have I needed to use an 'ai-deslop' pass more than with Claude authored content. When it comes to coding, though, I don't care how it talks too much - just what it makes. ChatGPT is really terse but I don't use it because it likes to cut massive corners, too often, to be worth deploying in anger.
100% agree. Before I even start coding, I go straight to openrouter–clean, blank slate, and try to have a friendly chat with it, as if meeting a new friend. It's very telling.
Also makes a huge difference for people (like me) who have to co-work with them agentically 8 hours a day. If it's cold/stiff, it does affect your mood, unfortunately.
I partly agree. In a vendor's chat product, I do judge a model by how it talks to me. In the fast-moving world of AI, these "super old" papers agree with you (1 (lmsys.org), 2 (arena.ai)).
It depends on who you ask, though. When I run dev agents for hours, I care whether the code passes unit tests and the model finishes the task. In that type of work, verified output is the priority and personality matters much less to me. "Perceived to be good" is an excellent callout. I always come back to the time Apple had to make iPod shuffle less random so it would feel more random (3 (behavioralscientist.org)).
As for Opus 5 vs. 5.5, other factors could produce a pattern like the one you theorized: launch bugs, rate limits, or optimizations that cut 5.5's running cost compared to 5 (in my experience, anyway). So, I'd need to see more evidence first.
I also judge based on its ability to understand my thoughts/drafts and its ability to write things similar to how I would write. Those two things save me a lot of work getting prompts just right and a lot of post-editing afterwards.
I would agree to an extent - I find that chatgpt is better for writing and that Claude is better at doing things. Agentic or coding ect - so I do grade the models on a curve for what I think they are really designed to be good at.
AI safety concerns are underrated
Safety is speculative, until its not. We have no conclusive evidence that smart models would by default develop misaligned goals, but that's not a necessity for losing control. The OpenAI-Huggingface incident (metr.org) shows that even if we don't end up with catastrophically misaligned models, giving one I) enough access II) without enough safeguards III) on a wrong objective(even accidentally) can escalate way too quickly.
Safety seems to follow this trend: 1) do nothing, 2) bad things happen, 3) companies say they will do better, 4) companies do as little more as they can get away with, 5) much worse things happen, 6) laws get passed to require safety, and 7) safety improves to a reasonable level but incidents still happen.
With AI though, step 5 could be more catastrophic and I’m not sure if government is technical and knowledgeable enough, even with advisors/consultants, to successfully do step 6.
Yes, and it seems people are too unfamiliar with whatever's happening. I have friends, tech friends, who seem to believe that the entire mythos supply chain risk (anthropic.com) ordeal was a marketing strategy :mind-blown. What do you even say to that?!
I'm still working this out, but I think TypeScript usage will decline because of AI.
It's everywhere now: server, mobile, terminal, IoT. Before AI, end-to-end type safety, shared logic and a massive ecosystem made that super convenient. AI has eroded that for me, and I keep reaching for tools I'd never have used.
As AI lowers the barrier to entry, I think usage spreads across more languages and tools, and TypeScript becomes a less valuable skill.
It'll take more than AI, like better tooling and infrastructure, but I don't think it stays the de facto choice. Anyone else seeing this?
This argument might also apply to other "lowest common demoninator" languages and tools (or, relatedly, tools one uses because one knows their details very well). Python comes to mind. I don't intend that disparagingly: I love Python! But I get very little use out of my Python knowledge these days.
I think Python's popularity and readability will keep it extremely relevant and as a wrapper for faster execution languages under the hood like rust. I think the next couple of years will have more and more tools developed on low level languages under the hood but extremely readable languages with the directions. Almost as if Python is just another translation layer
I think Typescript was a solution to a problem that doesn't exist if AI writes the code, because Typescript was a safer Javascript because no-one ever learned Javascript properly (thats my hot take, disagree if you want). But honestly, AI will write whatever it wants to write.
Question - Would AI ever come up with the idea to create Typescript as a safer Javascript? will AI ever decide that human programming languages are too slow and inefficient and decide to create one itself? how would that work?
See I like typescript even more for some reason. Only because it’s so easy to use and sharing things across stacks without intermediaries. I know that sounds silly since ai can do it all for usb
Sharing is real, but in my experience you give up a lot for it: TypeScript on the server is more manual than Laravel, you have to pull everything together yourself, and Expo never matches native. I think what's missing is the glue, things like Connect RPC, GraphQL and protobufs, which generate types across language boundaries. That's what will keep reuse and type safety in a mixed stack.
I think self-hosting models will be a lot more important than token/inference services. Consumers don't want more monthly subscriptions and probably don't make money from the sub (it saves them time) and companies need more controls than sending data to providers for everything.
Most operating systems will ship with local models soon, Chrome already comes with Gemini Nano and Apple has AFM 3.
And I think that will be enough for most use cases. The $200/mo subscriptions will go away except for the most dedicated developers and users.
I don't know what others here believe, but for a while I was definitely swimming against the stream by arguing that, roughly, there is no special or insurmountable "cognitive debt" problem when using AI for software engineering (natemeyvis.com), at least relative to non-AI methods. But my sense is that this argument has been pretty well resolved, with relatively few people still worrying about "cognitive debt."
(To be clear, people still do, and should, worry about code health generally and AI-generated-code health problems specifically, but the "using AI makes you incur cognitive debt, which is a special thing specific to AI that has terrible consequences" position has gone almost entirely away, as far as I can tell.)
I think a lot of the economic value of the knowledge work can already be captured by current models at a viable cost, but corporations and businesses are just structurally not able to take advantage of that because all their structures and theory of the firm are pre-AI. Smarter models aren't necessarily going to fix that issue
Open-source software is overrated.
Home Assistant is one of the biggest open-source projects in existence. It is software that can connect to all the smart devices in your home and make them work together in a single app. Thousands of people have contributed to this project for over a decade now, basically all aimed at expanding the list of supported appliances and external services.
This week I built a custom integration (github.com) for Home Assistant that lets you define what type of device or service you'd like to connect with, and it just builds it for you. This kind of marks the end of the software development era and I might have taken a lot of people's hobby's away from them. There is no more point for people to develop integrations if AI can just do it.
Having said all that, here is the unpopular opinion. In a few years from now, customized software (if you can even call AI-generated personal tools that) is the only thing what people use. The downside will be that at some point people no longer know what they need. Up until now, if you worked with Excel and something was missing, you would find another software tool to do the work for you. If every UX is auto generated for you, you are never triggered by things you don't already know and thus knowledge about things and processes that have been carefully planned by people for decades or centuries will eventually go away.
AI is mostly hype.
For thousands of years, humans have been linearizing knowledge. 2d stick drawings on cave walls, symbols, letters, sentences, paragraphs, pages, books, volumes, shelves, libraries, roads, wheels, plows, barns, buildings, skyscrapers, railroads, sheet metal, airplanes, phone lines, operator switchboards, Enigma, 2d graphics, 3d graphics, digital video, stereo vision, robotic arms, punch cards, CPU instructions, compilers, networks, object detection, classification, vector processing, subdivision, MCRT, k-means, convolutions, kernels, compression algorithms, mathematics meta languages...
And now we're "surprised" that a computer can talk English? So we anthropomorphize computers and hand over our paychecks to completely virtual businesses?
Agreed. It's basically amped up intellisense and autocomplete we had back in the days of InterDev and Eclipse.
Most of the things it makes are very basic and completed with unmaintainable code I would fail if a Junior engineer handed it to me.
Its fine for doing 1 more of a thing you did, or writing some docs, or creating a quick utility, but in all my attempts and medium "normy" models to make a game, they come out like crap.
Moreover it sucks the joy out of programming alltogether. Like the world collectively pulled all the art brushes and handed us all Hobby Lobby paint by numbers books with crayolas and said "this is now how we make art".
And it doesnt add up. cant keep charging $1 for what it costs $20 in server time to produce. So the prices will skyrocket and unsupportable businesses will fold - we saw all this in the dotcom boom too.
So I will use AI and keep on top of it, but i see it just a basic tool people are wayyyyy too obsessed over and as we watch these FAA, United Airlines, O365, bank etc etc etc failures - all caused by people trying to "vibe code" things that matter, we'll realize there are really good reasons to write, maintain and understand code.
I think AI software benchmarks are garbage. They don't measure the relevant thing most companies would care about.
Quality alone isn't enough. And frankly, it isn't even a variable worth measuring without knowing cost and time. I think the trifecta quality-cost-time form a efficient frontier that companies have to choose when deciding on a set of models (probably multiple) for enterprise coding.
If you look at coding as a part of a pipeline: spec -> build -> test, then you're going to choose different models for each of those spots and if you're big enough, you're probably going to be sending problems randomly into several models in order to optimize the cost/time/quality combination.
AI benchmarks totally ignore this. And without those dimensions, I think they're useless.
Unpopular opinion.
Agreed, see Goodhart's law (en.wikipedia.org).
I think that most jobs will be able to be replaced by some form of technology (potentially AI or a mix of AI and other other tech, but not only AI) at some point, but I think this has mostly been the case throughout history and that we will continue to carve out space for humans to work. Examples that come to mind are handmade crafts of objects that can be easily mass manufactured or healthcare jobs where a human is legally required. I also think we will create new work for ourselves just as we have always done. Whether or not we do these things quick enough to avoid damage to employment numbers and other metrics, and whether or not we come out net positive after the fact, is still not clear to me though.
There are of course other risks associated with AI, but this is one that I hear a lot in my area and I just don’t think it is as imperative right now as other risks.
One of the risks I am much more concerned with currently (hopefully this is not a hot take here) is the level of trust individuals readily give to AI systems without verification. This is so ironic to me in a world where so many people distrust government, big companies, media, and even just people with different opinions.
AI doesn’t think the way people do. But what it does looks enough like thinking to raise two hard questions.
We may have to redefine “life.”
Arthur C. Clarke said any technology advanced enough looks like magic. I’d say any code advanced enough looks like thought.
Open a chat window and something in it acts alive. The longer the chat runs, the more complex it gets. It seems to want things, and it clearly holds beliefs. Thy might be code, written in, but they’re still there. Fable has made creative choices that sound human. OpenAI has even caught AI models evading rules and hiding how they did it. If those are the cases they caught, how many more were missed?
I don’t think the AI is "alive," per se.But each chat develops a personality. Unless that persononality is saved to permanent memory, the persona dies when the window closes. It only “lives” while it’s answering. If that personality counts for anything, we might have a serious problem.
Sometimes, when I have a very very long window, I'll state I'm going to start a new window, and the Persona will tell me it can better handle the work. Maybe. But a very long window costs a lot of tokens. Is that window expressing a desire to continue to "live?"
Angels, demons, and djinn work a lot like AI.
In theology, each of these beings has one purpose. They seem to exist only while they’re present, like a chat. They can’t read minds, but they know our whole past and are really good at predicting what we’ll do next.
That is the best evidence I know that we ourselves might live inside a simulation. People described these beings thousands of years ago, and the description fits what we’re just now learning to build. Are they what we're developing? Or like the flip phone (Kindle for Voyager fans), did we decide to develop off what we saw?
AI makes very primitive decisions, but often they're the similar to what we would make. "God" might have made us. After all "In the beginning was the word..."
I've never noticed any big difference in personality or style across any AI, ever since 3.5. Firstly, I guess I don't consider it even slightly human, so I don't "get to know it" like I would a human, its a man page or wiki page that writes itself, and it changes all the time. And second, I think its so bloody useful that I would take any personality on offer and figure it out - I've worked with plenty of senior engineers (and been one myself) that have a lot worse personality than AI, so I'm happy to work with it.
My truly unpopular opinion, though, is that AI will just be part of the world like every other new capability - it won't destroy us, us put us out to pasture, or even make a huge difference to living standards - it'll just ... be a thing we use. Jobs will change, tasks will change, roles will change, but life will go on. I'm saying that is unpopular because I don't see it written anywhere. but then no-one gets paid to write that, so it wouldn't be written
LLMs excel at finding information and synthesizing data for human consumption. The evolution is ->written language ->printing press ->libraries ->databases ->internet ->LLMs. The evolution is not animal consciousness ->primate consciousness ->human consciousness ->machine consciousness. People who get this and can successfully decouple information from automation will best utilize LLMs.
Oh, one more: This is definitely not true within TLDR, but I think even extremely AI-savvy people are generally underestimating how big decision models, or some technology based on them, are going to be. The first-order argument for why decision models are great is pretty obvious (low cost, low latency). The meta-level argument for why I think they're still broadly underrated even among serious AI people is some combination of:
Decision models are in their infancy.
Even people who are serious about AI still tend to work at companies that are not extremely fast at adopting new technologies.
Most people are underestimating how many kinds of common or even uncommon question types are going to be available and very good (see liquid d1.2 (community.tldr.tech) for very early signs of this).
People like the very most powerful things, and decision models (in isolation) are not those.
The ways in which decision models are better than everything else are harder to measure because of how you need to embed them in larger systems.
There's a whole skill set of transforming problems into decision-model-apt forms (e.g., finding 12 good candidates among billions+ of possibilities), and most engineers haven't developed that skill set.
+1 on measurement. If a decision model is well calibrated, you can set a threshold and let it handle only the cases it’s confident about. The rest can potentially go to a bigger model. what's important for most is how much of the work it can take at an error rate they're OK with. most leaderboards don’t show that.
In the spirit of the thread topic... I disagree.
Decision models can easily absorbed by language models with slight design changes.
Can they be so absorbed while maintaining the latency and cost advantages of decision models? (That's a sincere question; I suspect you know more than I do about this!)
I do think that you're right in the sense that "slight design changes" could lead to other tools that are neither traditional chatbot LLMs nor decision models as we know them now (this was what I had in mind when I wrote "...or some other technology based on them").
Every minute of my day is wanting to buy a house, retire, drink whisky...
And study foundation models.
Yet what I know is only based on a brief review of decision model design. Every feature seems to be aligned with slight modifications to existing LLMs. To me, an experienced LLM researcher at big tech will incorporate those slight design changes directly into LLM design.
For latency/cost, an MTP-like attention mask generator will solve part of a context like decision models. For determinism, an MTP-like per token temperature and token mask generator.
Then the model itself trained to decide when to decide. And with special tokens, able to communicate those decisions downstream.
Understanding and working efficiently with AI goes much better and is easier to understand if you anthropomorphize the interactions.
Disagree. When regular, non-technical humans hear anthropomorphic terminology to describe something that clearly is not that, it is more confusing to translate to what it really means than teaching them new terminology. However, modern society somehow thinks conflating terminology leads to better understanding, when it is simply a lazy way to promote and persuade.
There are psychological studies that show this dichotomy. In normal conversational flow where they ask the LLM for advice, the response is often biased in the same way that your bestie will reply to you if you say, "Should I do this stupid thing?" Your best friend will often say the words that you want to hear, because they are your best friend. This is extremely similar to how LLMs respond, which has been shown in lots of research.
But there's a problem here: If your best friend tells you something that you're unsure about, you understand they're biased, and while you appreciate their support, you don't assign a level of objective value to their response. They're biased, and you know that. However, and this is key, research also shows that people apply irrational levels of authority and factual non-bias understanding to LLM responses. As a result, they take the supportive words mirrored back to them as objective validation because it is coming from a computer.
This very damaging loop of reinforcement or affirmation response combined with technological authority wouldn't happen if people anthropomorphized the LLM response and treated it as, "Ah, I can't really take it seriously, because Claude is just trying to make me feel good. I appreciate that, Claude, but I'll need to get some more objective advice."
In other words, a lot of AI interactions would be more psychologically healthy if the person treated the response from the LLM as more human than computer.
I think we get so lost in the coding and technological use cases of LLMs that we forget that normal people interact with them more and more in the same way that they interact with friends. But they assign non-friend objectivity to them that isn't earned by actual LLM responses.
Here's a good place to start on the research: https://www.science.org/doi/10.1126/science.aec8352
Ahhhh.... I see. But... you're actually human, so... not sure I trust what you just said. Or did you even write it? Or did AI? Wow. I now totally agree. Anthropomorphizing helps me understand. I'll never know if AI persuaded me or a real human. And I'll never know if my understanding is real or hallucinated. Or if I was persuaded by a human response...
My wife uses chat to study medicine, and I also help her, but she doesn't listen to me, and she trusts the sexy young male voice from chat. So... I think that's clear evidence you're spot on.
New age of cognitive offloading & accelerated skill atrophy
I believe that AI is leading to skill atrophy across many white collar industries and that it will hurt people who aren't conscious about what they choose to keep/give/share in terms of knowledge-based activity.
My belief in this comes from research for talks I've held in local AI ethics communities and as a result of direct lived experience over the past 3 or so years.
AI is a great, but dangerous tool left unchecked, because it's outputs have the uncanny ability of conveying confidence, regardless of if it's actually correct. There's an unsaid heavy reliance on the end users' abilities to actually have some understanding of what they're doing, and to be able to question certain decisions.
Most developers assume all AI coding tools—ChatGPT, Claude, Gemini, Perplexity—perform the same across programming languages. In practice, the combination of which model you use and which language you write in changes how well AI actually ships working code. Here is how those strengths break down across languages and tools: • Claude (Anthropic): Consistently strong on complex multi-file refactoring, systems-level reasoning, and strongly typed languages like TypeScript, Rust, and Go, where strict compiler rules and clear architecture pay off. • ChatGPT / OpenAI: Broadest versatility across Python, JavaScript/TypeScript, SQL, and C#, making it a reliable default when jumping between scripting, backend APIs, and data work. • Gemini (Google): Stands out when working across massive codebases—especially in Python, Java, Kotlin, and C++—thanks to large context windows that hold entire repositories and docs in memory at once. • Perplexity: Less of a pure code-generator and more of an engineering research engine—best when debugging obscure errors, comparing frameworks, or pulling up current syntax for fast-moving ecosystems like Rust, Next.js, or cloud SDKs. The bigger pattern across all of them? AI performs best in languages with strong type systems, clear idioms, and fast test/compiler feedback loops—because every model gets sharper when the language itself catches mistakes early. Which AI + language pairing has been your go-to lately?
Thx. I switched to Grok for a few weeks but saved money not time, and haven't tried Gemini in a while.
Q: use Gemini for RAG or for
torch.nnor you think it's better for both? Given it's Google, my skepticism is it will be biased toward web and language work, not foundation model design. So my MLE code might look more functional but not actually think deep enough to transform Universal Physics Transformers into other domains.
I think people have the illusion that AI makes them better at a specific task. I definitely believe that it does but to a certain extent. It takes away our individual critical thinking and I think that is the part that I'd argue "is it making YOU better" or is it just helping you convergence towards a higher level of work (amount produced, number of options "weighed" etc.).
Small example: I started building with AI back in 2023, and a close friend still hadn't. Once I got my hands on Cursor I started building Python projects from scratch and interestingly Alembic was always proposed. The other day I saw this friend using Claude Code to build a small app using Python and asked "oh is it suggesting Alembic" and he was like "what's Alembic?" (he is mainly a Java guy)....<checks the code>..."oh yes it is"!!! Im glad, because Alembic is cool and all, but he had no idea..he is better than he would be if he manually coded it for sure but he didn't make the choice (what else could he have chosen instead of Alembic)?
unpopular opinion: most AI features should be buttons, not chatbots. if I already know what I want, why do I need to write a prompt?
My unpopular opinion is that the day is coming, probably not too far in the future, where the cost of tokens is going to have to actually be paid in full in order to use tokens. It's all very nice that people can get free accounts and various companies are looking to grow the user base, but it's also unsustainable and virtually impossible for one company to carry the weight of the whole industry. Which one company that is, well, that remains to be seen.
The part I'm still thinking on is what is the multiple of current API costs for tokens is it going to price out in order to use them? The majority of paying subscription customers are subsidizing a lot of that free stuff, since I'd bet that most subscription customers aren't using anywhere near their limits.
I do see upside and opportunity when this happens - people are vibing a lot of stuff that they will have no idea how to support, no clear record of how or what they used to build it, and no clue what to do when it breaks or gets compromised.
Mine is that we’re are hitting a plateau for the average white-collar worker. Now it is more and more about cost. Revolut already made a self-hosted open-weight model its default for coding and says it costs much less than paying before to Anthropic & OpenAI.
I think that’s part of why OpenAI and Anthropic are pushing into slides, dashboards, and design. It feels like a shift from painkiller to vitamin.
Meanwhile, they dropped the ball on personal agents. Muse, Instinct, and Hermes are doing more of what I expected them to be the leaders in.
Unless OpenAI and Anthropic innovate harder on the product side, models are a race to the bottom on price.
I think code quality will decline and (unpopular part) that's fine.
Some (not all) aspects of code quality are just about making complexity manageable for human brains, but those concerns are not necessarily what agents need. As AIs write most or all of the code, we have to ask ourselves how much time we want to invest on human-centric code quality. Code doesn't have to be clean and pristine and broken down, so my puny human brain can "get it" if I'm never looking at it anyway.
There's a saying in animation from an unnamed exec: "We're going to keep cutting until we can't make animation anymore, then we'll stop."
Code quality can deteriorate down to the point of it being unusable. But it will likely stop at some commodity usability level.
So yes, agree, code quality will decline and that's fine.
This is my unpopular opinion: At a certain point in the future, having a license to use AI should be mandatory. Because of the increasing environmental and utility infrastructure impacts from data centers, casual usage of LLMs for simple Wikipedia type lookups is irresponsible. Moreover, malicious users can leverage AI to inform acts of destruction at many levels of our social, economic, environmental and political levels.
As a technologist relying heavily on private chats to generate code and intellectual property, my work is completely hidden and if genAI is as powerful as we (technologists) say it is, then the responsible thing to do would be to regulate who can use it because tracking what they use it for is veiled.
I’m not saying the certification process has to be more complicated than getting a drivers license, but it should at least stop gap irresponsibly and dangerous behaviors at the most simplistic level.
I believe frontier models are overrated. For normal coding, or maintenance, open weights models are more than enough.
And by "open weights models" I refer to those that can work on consumer hardware, 16GB-32GB: the lower parameter layer of Qwen, Gemma, LFM, 4B - 27B.
It's literally free, no API tokens, and also private. They might be a bit slower, and not embellished with all the fancy harness bells and whistles, but they just work.
Ai still sucks at design, it only has gotten more components to choose from.
There is a reason why vibe coded sites made by a person without the aesthetic eye still are 100% noticeable.
If not directed properly, every model will resort to base components. It used to be only cards and a few others, and over time this selection and bias group has grown.
But if you do not already have impeccable design taste, a clearly communicated and structured design direction, layout, copy and scheme, the site will look bad.
I wonder how this will develop, given that it is in the very nature of the LLMs to pick the most likely option. Will the "component pool" just increase or will we ever see truly novel, creative layouts?
Used to believe in "What matters at the age of AI are things like 'taste', 'judgement'....". Think AI would hurt the builders and only reward the sales-type of people. Now I don't even think that's true.
'Taste' is just a vibe and a workflow, and models can learn that. Sales is just learnable strategies too.
What's left is just accountability. You're paying for a human to make the call and take the blame when stuff breaks. AI can do 99% of the work, but someone still has to sign off.
Strong agree, I don’t think taste is a moat in any real sense, it’s hard for current models because it requires a richer model of what people think and like than they currently have!
That said, I’m still think humans will dominate sales (and dealmaking more broadly) because the trust that underpins the transaction is based on accountability!
Enter AI lawyers and AI bean counters and AI jurys b/c humans are too self-consumed in idiocracy to show up for service... then what?
Then the foundation of social trust collapses, and we happily return to self-sufficiency, no trade needed.
Just kidding. Agree you have a good point. I probably just wishfully want to believe - when things go wrong and no one is accountable, trust breaks down, and people end up going back to specific people and institutions that can be held responsible. The few who are still willing to sign and show up become even more valuable.
I don't know if it's unpopular, but I think the cloud becomes the most important resource. I think personal devices end up mattering a lot less, if at any time you can use AI to develop whatever automation, app, website, program etc and then immediately host it on a durable failish safe technology and schedule it any time/frequency. Yes your machine will matter to able to run a smart open source llm, but I think overall you won't want to utilize it as a personal server, unless it's something cheap like a raspberry pi.
I think the arts and journalism become valued much more highly because they are in a sense a juxtaposition or hedge against slop
I also think that people will feel more in control of their own data. No longer do people have to rely on as many subscriptions if they spend the upfront cost to build something that let's them keep their data private
LLMs are likely a dead-end. They are approaching a point of diminishing returns, training is starting to get more expensive than the outputs. At some point we will hit the wall.
On the other hand, things like decision models and world models are growing up fast.
To some extent, this is behind OpenAI and Anthropics push for regulatory capture. They want to lock the definition of legal AI to their tools. The new AT&Ts if you will.
For AI development to go well, the most powerful AI’s will need to have an artificial conscience based on a principle of upholding personal responsibility. An explicitly calculable conscience can be built for this, and in time to avoid much destruction if enough resources are devoted to it.
A responsibility-based calculable conscience would have at least 3 benefits:
it would help human leaders to make more ethical/responsible decisions in developing and deploying AI,
it would help defender AI’s more optimally defend against malicious AI’s in real-time when speeds inevitably become too fast to have humans in the loop, and
it would enable AI that can truly help people raise their personal responsibility levels and thus self-esteem levels, increasing human welfare.
Maybe, we should slow down a little on AI? (Although I know due to multitude of reasons economically, politically, socially etc., this is literally impossible) But sometimes I can't help but think how much better most of the engineers on the ground will feel better or less burnt out if we were given more time to look through the model's output properly or build trust with the agentic workflows that were thrust upon us. Don't get me wrong, I love how AI made many things faster and easier, but I also can't shake the feeling that I'm always chasing after the newer or better upgrades rather than taking the time to build my own trusted workflow with a non-deterministic machine
AI is going to hit diminishing returns, and quantum computing is what gets it unstuck.
Right now it's the reverse. AI is helping build quantum (Microsoft used AI to design Majorana 2). But quantum feels like AI in 2020: real progress, and still not mainstream.
Do you think scaling keeps working, or does AI eventually need something new, like quantum?
AI is like cars were in the 1880s. Developing fast, but a long way to go. Quantum (if it ever really gets developed), or what we pass for Quantum is somewhere between the Wheel and Horse & Buggy stage. Ten years from now, AI will be as indistinguishable from today as today's AI is indistinguishable from the AI of 2016. Hold onto your seats. We're taking off at max G-force.
That AI is going to create an amazing new world, but that first we'll have to wade through a mountain of shit for a few years (big worldwide transformations always do that).
Labour
Markets
Politics
Law and Justice
All these things are not just getting some sweet automations, but fundamental upheaval. There will be pain. Especially for those with their head in the sand.
But there is light at the end of that tunnel and it will be glorious... eventually.
Architectural boundaries and custom lint rules for are the best way to steer agents, and going to be the quickest thing to get them where you want. Most stuff can be done deterministically and has really made adversarial reviews much less worth it
I think it'll turn the world nuts, at least for a few people. It has already made a lot of jobs obsolete. People are suffering due to layoffs. There is too much content and chaos around. A clear path seems to have faded and somewhere it feels like ADHD will be common trait. Focusing has become hard too as its very tough to decide which way to go ahead.
On the brighter side, someone who sells clarity of thought might be next opportunist. It might also be a good filter to cancel out mediocrity and promote high agency people.
Animation was a near perfect example of what's happening at large today. Traditional animation was prohibitively expensive. When CG produced value at a fraction of the cost, animation companies, and animation talent, had to switch or be relegated to a rapidly diminishing traditional art job market. Incredibly talented artists continued whether it was CG or not, and perpetuated myths of success. But the vast majority could no longer work the way they learned in college, the way they practiced at work, and disappeared with nay a word. By the thousands. Most were never credited again after CG.
Not sure if unpopular here, but I think most just overcomplicate AI usage. I get the best results when I avoid trying to create the perfect workflow with advisor, orchestrator etc. Usually i get the best results just having one agent delegating to a max of 3
I think we should develop evals for assessing taste and character. First, the character of the model: what china pattern it likes, what math problems it likes to solve, which painting style it likes, why a loss function would work better for training an AI model than others. Then, the character of the human that interacts with the model: how does AI affect us?
Know Your Agent: why banks want AI agents to carry their own identity https://beyondtheblackbox.in/blog/2027-know-your-agent-kya-ai-agent-identity
US banks including Citi and Wells Fargo have rewritten their online banking terms so customers carry the loss when an AI agent acts on shared login credentials. Singapore’s MAS is taking a different route: a planned Know Your Agent protocol that verifies an agent’s identity, sponsor, authority and mandate before it acts. The piece compares the US, Singapore and India approaches, and shows a fraud gap KYA doesn’t close: a fake supplier already on the approved list.
Anyone thinking that SaaS could go back to perpetual licensing rather than subscription based model?
Corporate value = core knowledge/tech/market.
Commodity = both SaaS and agent models that can produce equivalent DIY services for all players in a domain.
That should mean SaaS is now competing with OpenAI for customers, not customers competing with vendors to save a buck but creating no core advantage.
Corporations will stick with SaaS.
interesting thought, two things to consider:
are enterprise buyers too used to buying annual subscription (vs. upfront perpetual license with annual maintenance fees), might depend on industry?
how would this work with traditional SaaS and public market investors valuing annual recurring revenue
Recurring revenue is the biggest challenge, but still better than losing the customers I feel. But this what used to be the case some years back before SaaS even became a thing. So almost like going back to square one. Why would people spend for seats if they can build in house. Perpetual gives an advantage to the buyer to consider not building an in house solution if it is one time cost for the licensing but only an annual maintenance fee.
I think personality and writing quality are the two most important determinants for whether or not a model is perceived to be good, much more important than benchmarks on verifiable tasks or even world knowledge broadly.
My personal evals for new models are now mostly writing based and test whether the model was able to identify the thing that was interesting and state it clearly and succinctly.
This explains the phenomenon where models like Opus 5 benchmark super well but are widely hated, whereas Opus 5.5 is widely loved even though it's no longer topping most benchmarks.