Seems pretty...underwhelming as the first product of x.ai considering the initial announcement of its founding came with such a splash. This is more like someone trying to piggy-back off the popularity of LLMs with a trivial tool than driving the space forward in any way.
It is also in that sour spot where it is completely closed, which might be fine (GPT-4, Claude 2) if it was much better than the open-source models, but that does not seem to be cut-and-dry: https://huggingface.co/openchat/openchat_3.5#comparison-with...
If this stuff is so powerful, or going to be so powerful, is positioning like this wise? It kinda feels like it is aiming squarely at its own obsolescence. That is, if LLMs turn out to indeed be "the way forward," than surely the esoteric/specialized science of prompt engineering will fall away precisely in step with how much better next-gen LLMs are, right?
Or to put another way, what is the ultimate value of LLMs in general if they are not aiming to make the very idea of "prompt engineering" obsolete?
If you want standard responses then presumably LLMs will start to converge on those responses. But if you want a non-standard response then you have to ask for it, and I think that will continue to be a kind of "prompt engineering".
For instance I'm working in math problem generation right now and you could imagine saying "make the problem harder" and you'll get back... something. But what does "harder" really mean? It's very underspecified. If you work in making educational material you probably have some much clearer criteria, and if you are building a product you may have a clear point of view about what you want in the next problem in a sequence... and "harder" is just punting on all these details.
The tricks may stop being as necessary, but the deciding what you want and how to explain what you want seems essential no matter how smart the LLM is.
If you were to talk to a human that has enough context in the problem space, telling them "make it harder" should achieve something that makes sense... easily, no?
I think that at some point a generically trained LLM will be able to do just the same. Prompt engineering will probably be a niche for security (both blue and red teams).
I understand what you are saying. When OpenAI's API was still new, there was a LOT of "prompt engineering" required - a lot of proper formatting and tricks to get it to produce the required output, proper ways of formatting prompts with "few-shot" examples, etc. Packages like LangChain sprang up because trying to chain together multiple prompts with different inputs and outputs was becoming a bear.
But then, within a few months, the LLM's got better and better to the point where the average ChatGPT user doesn't need to be a "prompt engineer" to get what they want. Sure, there are tricks with adding specific phrases to a prompt to get it to do a better job, but what we now consider "prompt engineering" and what was "prompt engineering" pre-ChatGPT are very different.
LangChain, for example, has reacted by pivoting away from just prompt chaining and templating and into an ecosystem for agents and automated AI "bots". I think that may be overkill for many use cases, and there are good alternatives (Rivet is now my go-to) if you still want to build an LLM-powered app that just requires a specific logical path and specific inputs and outputs, but honestly if I was working at an "AI startup" I would probably be trying to focus my time on getting away from "prompts" and more just to "doing a good job of what you are told".
I do think that "prompts" (especially chat-style conversations) lack a lot of the nuance that you need for a production-grade app especially when it comes to specific input and output formats, etc. Function calling helps a lot but I don't see why xAI is focusing on an IDE for writing a great prompt instead of building an LLM that can make good, structured output with just an "ok" prompt.
The next year will be interesting - it hasn't even been a full year since ChatGPT was released, and look how far the industry has come since the closed-beta release of GPT-3.
For me, I would just like a simple way (no technical knowledge) to invoke LLM sub-steps. Exemple "Find the 4 articles of laws most relevant to this case, then anaylse each of those articles separatly". So, a simple non-technical way of doing auto-gpt style substeps. This would greatly increate ChatGPT precision.
I suppose it's always a correct decision to vertically integrate and invest heavily in tooling, from a pure monopolist standpoint. Doing so reduces stakeholders and improve control, which should grant more power to management than constantly having to succumb to engineering, market or baseline reality.
But e.g. very few laptop manufacturers are even capable of designing and fabricating own display panels(not sure Apple does?), let alone ones that provide any advantages, so there's that.
Others have commented to maybe help with my bad writing, but: I am basically trying to say that the promise of LLM models are in the long term at odds with the idea of "prompt engineering" even being a thing in, like, two years. Every improvement to existing models can only really make such a thing more and more obsolete.
Otherwise, what are we even doing here? Expensive regex?
It’s the sort of thing an engineering team builds when they have way too much time and no idea what to build. There is zero reason to use something like this over, say Jupyter notebooks and the advantages of notebook are obvious: actual researchers are already familiar with it, you can integrate everything into your data science workflow, visualise results etc
I have serious concerns about anything from X at this point.
At the risk of going into some amateur psychological analysis of Elon Musk I think he's pretty well past the point of what happens to so many people insulated by wealth, power, influence, their carefully crafted bubbles/echo chambers, etc.
For example, for all of his talk of bias I am completely unable to get positive and glowing reviews of Tesla stuff out of my X feed. Every single day I label the non-stop Tesla, Cybertruck, posts from Elon himself, etc as "Not interesting to me". I've never given the algorithm any reason to think I want this stuff repeatedly shoved in my face and actually quite the contrary as I've said.
I've been doing this for at least a month and yet they persist. It's pretty clear he has his "thumb on the scale" (to put it mildly) for his own ample self-interests, ego, politics, etc.
Plenty of people and platforms are of course prone to the same issues but from what I've seen this is so gallingly obvious with him and X it's borderline insulting to those of us that have a modicum of understanding of what's going on here.
I think it's very highly likely this will be reflected in anything he or X produces.
I was going to say that I never saw anything like that, but I remembered I followed Musk's advice the day he recommended it: I disabled recommendations and only use the "follows" view.
I dont have that experience at all, and im a tesla owner. I see more negative tesla stuff than positive, especially about stock value and cybertruck quality problems.
I think clicking "Not interested" button is optional. You can just straight up block the content, and adblock any "You might also like".
This is my bigoted view, but my interpretation of "you are the product" principle in freemium model is that they pay you in perceived value. Not the other way around, nor in cash. So, onus is on them to bring you values you recognize.
Unfollow people that follow him and you won't get this content. My content used to be all over the place as well. After I've done the pruning, now it's various people talking about topics that I care about. Obviously it's not perfect - things leak through, but far better.
Same with Facebook, my feed is now pretty much photos and news from friends and family.
> This is just an ad hominem pretending to be civil discourse.
The closest I got to an ad hominem was saying he seems to be impacted by a common problem that afflicts the extremely wealthy and powerful. It doesn't take much research to find celebrities, the ultra wealthy, etc talking about how easy, dangerous, and seductive it is to become surrounded by sycophants. I don't know of anything else to attribute his increasingly bizarre and erratic behavior to. Although some people have gone as far as to say he has "friend his brain with drugs", which I think absolutely is an ad hominem.
In terms of XAI, my experience with the Twitter algorithm is clearly relevant as it's obviously an AI implementation at X. If you want something more direct, Elon Musk has said XAI is "based" and "loves sarcasm" while implying it was directly modeled after him and his own sense of humor. From[0] citing his direct announcement:
'"It's also based & loves sarcasm," he wrote. "I have no idea who could have guided it this way."'
Creating an LLM with a "sense of humor" ranges from "interesting choice" to absurd to dangerous. Humor is a matter of subjective and highly individualized tastes, and sarcasm can be (and is often) lost on people, which starts heading towards dangerous. "Based" is meaningless but also implies it has been influenced towards whatever he thinks that is, which I'm guessing is more alignment with him and his own opinions, etc.
I have to imagine if Sam Altman, Mark Zuckerberg, etc were to attempt something similar someone around them would tell them it's a bad idea (because it absolutely is). Announcing it, touting it as a positive, and being proud of it is next-level.
The world does not need eccentric tech billionaire mini-me AIs and the fact he seems to have created one speaks to an ego run amok.
This is easily explained but not in a positive way. X Subscribers get up-ranked in replies and For You. 99% of anyone who would subscribe to X are Elon Fans. As a result, you get the fanboys.
That's absolutely false. It's trivial to prove that OpenLlama is both more transparent (details on how it's trained), and more open (better license, and public training code).
Llama 2 doesn't even mention the dataset used, so it's not very transparent.
even if this is true, to be fair to elon he donated to openai when it was a nonprofit and through some sketchy moves it was turned into a for-profit company where his donation should now be seen as an investment
To be fair to OpenAI, Musk determined that OpenAI had irreparably lost the race to Google and proposed that he be made CEO [0]. Musk's public complaints about OpenAI being profitable were years[1] after he had decided to abandon OpenAI.
> But in early 2018, Musk told Sam Altman, another OpenAI founder, that he believed the venture had fallen fatally behind Google, people familiar with the matter said.
> And Musk proposed a possible solution: He would take control of OpenAI and run it himself.
> Altman and OpenAI’s other founders rejected Musk’s proposal. Musk, in turn, walked away from the company — and reneged on a massive planned donation. The fallout from that conflict, culminating in the announcement of Musk’s departure on Feb 20, 2018, would shape the industry that’s changing the world, and the company at the heart of it.
