To make a LLM relevant to you, your intuition might be to fine-tune it with your data, but:
1. Training a LLM is expensive.
2. Due to the cost to train, it’s hard to update a LLM with latest information.
3. Observability is lacking. When you ask a LLM a question, it’s not obvious how the LLM arrived at its answer.
There’s a different approach: Retrieval-Augmented Generation (RAG). Instead of asking LLM to generate an answer immediately, frameworks like LlamaIndex:
1. retrieves information from your data sources first,
2. adds it to your question as context, and
3. asks the LLM to answer based on the enriched prompt.
RAG overcomes all three weaknesses of the fine-tuning approach:
1. There’s no training involved, so it’s cheap.
2. Data is fetched only when you ask for them, so it’s always up to date.
3. The framework can show you the retrieved documents, so it’s more trustworthy.
This is the state of LLMs today - it is likely that we will have models in the future that can do some form of "online" training - or new training methods that aren't nearly as compute intensive. There are many people working on these scaling issues with LLMs today. We already have new attention heads that work around the quadratic time and space complexity of the input prompts.
Spend seed round investments on building a solid software but not building an income stream that can satisfy investors, thus not receiving any new funding and let the company die.
There's gotta be somewhere in the middle. Vercel's movements feel a lot like the "Embrace, extend, and extinguish" playbook.
Maybe there is a class of developer out there that doesn't get spooked by that but it definitely has created an adversarial place for Vercel in my mind. I feel like I need to be careful when touching anything Vercel have touched so that I don't fall into a trap.
If you have any concrete feedback on what we should improve, I’m all ears. We heard feedback from the community that they wanted better documentation and guidance on self-hosting and we shipped it last month[1]. Curious what you’d like to see improved.
I just stopped using their NextJS project because you can no longer self host the middleware, they now only support edge runtime and several libraries don't work with it.
I'm calling this situation Fauxpen Source. The recent moves definitely feel anticompetitive or at least trying to force you into using their products
I'm migrating to vite+vike (next/nuxt like experience for any framework)
Middleware does work with self-hosting[1]. It’s a more limited runtime that’s based on web standard APIs, which creates optionality for running it in high performance / resource constrained scenarios.
It _can_ work, but _won't_ for most real world workloads
Beyond the runtime limitations, it is poorly designed and requires you to effectively write a router when the rest of the system has automatic routing assembly
Not OP, in my case OpenAI does not want my money, they only accept credit cards. For example netflix wants my money so they have more choices.
Also I would like to pay for an equivalent alternative that is less censored, like ChatGPT had a bug one day that it refused to tell me how to force a type cast in TypeScript, it showed me a moderation error. So I want an AI that is targeted for adults and not children in some religious school in USA.
there is a somewhat unfiltered GPT4 at Azure, but they really don't want anybody's money (afaik only "trusted" corporate entities can access it)
at this time, your only option is local models. if you don't have the hardware to run them yourself, there are plenty of hosts - poe/perplexity/together etc.
llama3 is (hopefully) coming soon, and if it has improved as much as llama2 improved over llama1, and provides at least 16k baseline context size, it will be in between gpt3.5 and gpt4 in terms of quality, which is mostly enough.
Yes, there are 2 issues for me, my hardware is not powerful enough, only 8Gb vram and the models are still not intelligent enough. At this moment I have opened tabs for different websites and when I ahve a question I compare them and see the state. I would like a model that would say I do not know more often then respond with the wrong thing, also I would like it to follow instructions , now I ask them to "rewrite previous response but without X" and they respond "sure, here is the response without X " and they do not followed the instruction, like they are "hard coded" to do X. An example for X is "do not add a summary or conclusion"
Scrub to 01:54. That's a table of the flagship smartphones at that time. The last column of the table is the price in CNY. 3k, 4k, 5k -- around that range.
Now scrub to 03:20 where their CEO announced the MSRP. Listen to that 30-second-long "wow". Look at that price. The astonishment was real.
That's the price range of Shanzhai smartphones.
On a flagship-spec'd smartphone.
That's monumental in the history of smartphone evolution in China. Ask any friend of yours born & raised in mainland China. I bet none would argue otherwise.
---
Disclaimer: I moved to the US in 2017, and my experience since then with tech trends in China has been scarce.
The Shanzhai movement was also mostly based on pirated WinCE and the MediaTek all in one chip. After that became outdated (Android replaced WinCE, MediaTek lots its commodity advantage), they really didn't have much of a chance (I don't remember seeing many Android Shanzhai phones, but then I wasn't really paying attention back then).
A lot of the design team (almost all Chinese) I worked with at Microsoft China 07-09 moved over to Xiaomi to help found their UX design team (I see one is now a UX director at xiaomi, he was an intern in 2007), which I found pretty cool. They had previously done a lot of Windows Mobile work, and were really committed to trying out some of their wilder designs.
Oh yes, thanks for bringing up the pre-Android-era Shanzhai phones. Were they on WinCE? I was too young to get my hands on one for any first-hand experience.
They were almost all on pirated WinCE, not even windows mobile. It might have been all pirated WinCE, but an open sourced Android obviously killed all of that (circa 2010? I don’t remember exact dates but I had a friend working as an engineering manager on the Kin team who talked about it a lot at lunch).
Would you be able to spare some time and find us some references here? Despite a native Chinese speaker, I can't seem to gather up the correct terms on baidu.com
google.com to get me relevant materials on that pre-Android-era mobile OSes.
I’m pretty sure it’s Windows CE and not WinMo, most reviewers weren’t concerned about the difference. WinCE was the core of WinMo (well, until windows phone), but it was easier to just take winCE and skin it since they didn’t really want the WinMo UI (this phone has an iPhone copycat UX).
Pirated WinCE?? Do you have any source that those MediaTek software were CE based? I thought they were something more primitive, something like FreeRTOS.
> Part of the reason for this is the low investment in new technology by Shanzhai phone makers and their dependence on cheap and freely available chipsets from companies like MediaTek and Huawei running pirated operating systems, like Windows Mobile. Industry analysts believe the near future won't deliver a dramatic improvement of the usability of the pirate phones.
There really was no other option before android came out, since Symbian was never pirated (I think).
I think something must have been lost in translation then, most Shanzhai phones were heavily modified and skinned featurephones, not smartphones.
If that was understanding within MS China, I wonder whether that had an impact on MS' strategic decisions - whether they thought CE/WM has great market penetration in China and required changes from existing platform to WP is minimal...
I’m not sure what a heavily skinned feature phone OS would have been at the time, maybe Symbian. It was easier to take the mediatek chip and throw WinCE on whatever, even if you didn’t add many smart phone features.
It definitely made for a lot of jokes, but I don’t think the executives, at least on the MS china side where I was working, took it as a good omen at all. These were junk phones, and the iPhone at the time clearly wasn’t junk. That WinMo was shortly thereafter abandoned makes me think Redmond thought so as well.
Microsoft also locked down WP, so it wasn’t pirate-able, and since Android was already out a couple of years later when Wp was done and was basically free (no need to pirate!), well, everyone lost interest in whatever Microsoft was doing with mobile.
I believe it was some MediaTek demo code that could run directly on the modem for a 1-chip GSM featurephone or in a AP+BP 2-chip 3G configuration. Wasn't Symbian, nor CE nor Linux, more like a giant Arduino sketch. There were a lot more giant Arduino sketches / baremetal library-based big main loop RTOS in the '00s.
The 2009 article discussed here links to a PDF which is in Chinese and doesn’t seem to go into the software side much, but on page 20 you can see a device that appears to be running Windows CE. The Windows logo is visible on screen.
I didn't grow up in China, but had a Redmi 3 several years ago and can attest that the price was just too good to beat. Unlike the phone I bought in China back in 2011, the Redmi had a nice UI and felt like the closest thing to an iPhone of any Android I'd used at that point.
> In a sense, I feel like the shanzhai are brethren of the classic western notion of hacker-entrepreneurs, but with a distinctly Chinese twist to them.
Yes. The more I browse though the NAICS hierarchy, the more I feel uncomfortable excluding hardware-heavy companies like Garmin.
On the other hand, it also context-dependent. Let's say you're looking for a job opportunity as a software engineer/developer/programmer/..., how much priority would you put on whether the company has tangible inventory?
I think the ambiguity in the term "tech industry" is one reason related discussions flourish.
Thanks for sharing your story. Switching from a tech to a "non-tech" (if you allow) company doesn't sound like an easy decision. I assume the NPO must have a vision and/or a purpose that attracts you. Do you mind sharing some of the reasons that motivated your switch of career?
1. To work on a stable product. At startups, I never got to work on the same product through multiple versions across several years. I wanted the experience of launching something, then iterating on it. To become an expert in a market and develop long-term relationships with users or customers. All the startups I'd been in either failed or pivoted within a couple years, and all the consulting work I'd done with startups was always parachuting in to help on something and then leaving soon after.
2. As mentioned, to work on something more fulfilling than SaaS products. No shade on anyone who works on these, they are often really fun and exciting. But, my last job was trying to sell a yearly $60k subscription service to companies so that they could lay off members of their team and replace them with our product. My current job is helping scientists who are researching a very nasty disease that has directly affected my family. Not only can I explain what my current company does a lot more easily, I can honestly say I am there for more than the money or how good it looks on my resume.
3. Less work. My previous job had been described—by leadership, with a wink and a nod—as 'a pie eating contest where the prize is more pie'. That is, work really hard until we get PMF, then if we get that, work even harder so that we can get acquired. At the end of that, maybe my stock options would be worth something, but the owners and investors certainly would get a nice payday. I'm in my 40s now, I'm not going to work myself like a dog anymore, I kinda just want to have a nice, easy job, as long as I can feel good about doing it.
https://github.com/StarsRail/Cocai
Better than a human, it can draw an illustration of the scene in meme seconds.
It can also roll dices in 3D: Demo: https://youtu.be/8wagQoMPOKY?si=oCa2erHvheEyEKM_&t=55