So ChatGPT has a cutoff date on the stuff it can talk about. This predicting weather sounds like ChatGPT being able to predict next week's news from which it has been trained on. I can see how it can probably predict some stuff like Argentina winning a football match scheduled for next week when played against India given India sucks at football. But can it really give any useful predictions? Like can it predict things which are not public? Like who will Joe Rogan interview in 2 weeks? Or what would be the list of companies in YCs next batch?
Sure, not every model is an autoregressive transformer. And even a GPT could give some useful predictions if you stuff the context window with things it's been fine tuned to predict. We did that to get GPT to play chess a few years ago.
Specifically, I could imagine throwing current weather data at the model and asking it what it thinks the next most likely weather change is going to be. If it's accurate at all, then that could be done on any given day without further training.
The problems happen when you start throwing data at it that it wasn't trained on, so it'll be a cat and mouse game. But it's one I think the cat can win, if it's persistent enough.
I just don't understand how can your produce new knowledge which it don't have access to. Are you you folks claiming the future weather is a function of previous weather and the model is capable of replicating the function?
No one is claiming that there is "new knowledge" here.
The entire class of deep learning or AI-based weather models involves a very specific and simple modeling task. You start with a very large training set which is effectively a historical sequence of "4D pictures" of the atmosphere. Here, "4D" means that you have "pixels" for latitude, longitude, altitude, and time. You have many such pictures of these for relevant atmospheric variables like temperature, pressure, winds, etc. These sequences are produced by highly-sophisticated weather models run in what's called a "reanalysis" task, where they consume a vast array of observations and try to create the 4D sequence of pictures that are most consistent with the physics in the weather model and the various observations.
The foundation of AI weather models is taking that 4D picture sequence, and asking the model how to "predict" the next picture in the sequence, given the past 1 or 2 pictures. If you can predict the picture for 6 hours from now, then you can feed that output back into the model and predict the next 6 hours, and so on. AI weather models are trained such that this process is mostly stable, e.g. the small errors you begin to accumulate don't "blow up" the model.
Traditionally, you'd use a physics-based model to accomplish this task. Using the current 3D weather state as your input, you integrate the physics equations forward in time to make the prediction. In many ways, today's AI weather models can be thought of as a black box or emulator that reproduces what those physics-based models do - but without needing to be told much, if any of the underlying physics. Depending on your "flavor" of AI weather model, the architecture of the model might draw some analogies to the underlying physics. For example, NVIDIA's models use Fourier Neural Operators, so you can think of them as learning families of equations which can be combined to approximate the state of the atmosphere (I'm _vastly_ over-simplifying here). Google DeepMind's GraphCast tries to capture both local and non-local relationships between fields through it's graph attention mechanisms. Microsoft Aurora' (and Silurian's, by provenance, assuming it's the same general type of model) try to capture local relationships through sliding windows passed over the input fields.
So again - no new knowledge or physics. Just a surprisingly effective of applying traditional DL/AI tools to a specific problem (weather forecasting) that ends up working quite well in practice.
Thanks for the explanation. I am still a bit confused how this takes care of the errors? I can see how the weather prediction for tomorrow might have less errors. But shouldn't the errors accumulate as you feed the predicted weather as the input for the model? Wouldn't the results start diverging from reality pretty soon? Isn't that the reason why the current limit is close to 6 days? How exactly does this model fixed this issue?
It doesn't take care of the errors. They still "accumulate" over time, leading to the same divergence that traditional physics-based weather models experience. In fact, the hallmark that these AI models are _doing things right_ is they show realistic modes of error growth when compared with those physics-based models - and there is already early peer-reviewed literature suggesting this is the case.
This _class_ of models (not Aurora, or Silurian's model specifically) can potentially improve on this a bit by incorporating forecast error at longer lead times in their core training loss. This is already done in practice for some major models like GraphCast and Stormer. But these models are almost certainly not a magical silver bullet for 10x'ing forecast accuracy.
99.99% of the human population is in sleep mode. Zoning out constantly. Once you became awake, everyday experiences starts to become beautiful and a bit magical. Also you start to ponder more on your awareness and topics like eternity, god, death etc come to your focus than make believe stuff like careers, identity, nationality, opinions shared in tweets, the story you have about your life etc. Also a once atheist you starts to immediately understand the religious teachings.
It's hard to understand this statement if you are not awake. I would have down voted this before the shift happened for me :)
I second the G.I. Gurdjieff recommendation, but maybe start with any one of the books of "Psychological commentaries" by Nicoll, or "In search of the miraculous" by Ouspensky.
It happened to me by chance. But I was seeking it. I think the journey has just started for me. It has its own challenges. But waking up with Sam Harris is probably a good place to start. I don't think it's also for everyone. Life became easy day to day but became more challenging in terms of purpose etc as ego dies.
Unfortunately, maintaining a career remains necessary to prevent the local authorities who are in charge of money distribution and tax collecting from causing harm to my person and family. And the 99.99% of the other people who surround me, who seem to have a general dissatisfaction with identities, nationalities, and tweets, whether theirs or other peoples, are the ones who keep empowering the various authorities that keep this silly conflict based system running. It's really quite an undesirable situation.
RA Wilson says the only options after visiting Chapel Perilous are to become a paranoid, or an agnostic, and that there is no third way. I'd argue one could also become an atheist. The paranoid thinks there's always a force out there trying to get them. The atheist is convinced there's never a force out there trying to get them. I try to remain agnostic as I am not sure all the evidence is in yet to be able to make an accurate assessment of the situation which we all find ourselves dropped into.
I take modified Sturgeon's Law view of various religious teachings. Sturgeon says 90% of everything is crap. I think it's closer to 99%. But the 1% that's left over is pretty interesting.
Anyway I gave you a deserved upvote, but I'm sure your karma that matters is not as negative as your HN karma. :)
> Also a once atheist you starts to immediately understand the religious teachings
I ponder the topics you mention and at no point did I feel compelled to radically retool my life around 2000 year old religious documents which themselves have clear lineage to even older, less-informed belief systems.
That's okay. I didn't decide to ponder the religious teachings. After the shift I immediately started to understand the teachings that I once thought was primitive and irrelevant. They all point to the same thing. It's hard to understand what they are pointing to without knowing or experiencing it. Read a bit about perennial philosophy. But it's a bit like telling to blind person what colors are. It will not make sense till you see.
Less abstractly and more concretely: What do they point to? What are you claiming is the unifying deep truth beneath the surface of all religious texts? What heuristic is used to differentiate a text as religious versus the babblings of a schizophrenic?
I was raised Catholic, by a deacon, and have a deep understanding of Judeo-Christian religions, and have studied to a lesser extent several world religions. What am I missing?
I am just a student of these things myself so take everything I say with a grain of salt. But based on your question and my own experience, it seems that you are approaching religion in the same way I was doing -- intellectually.
That is natural, but the problem is that our intellect is quite limited. You would like to have a description of "the unifying deep truth". But even very simple experience, like how strawberry tastes like, cannot be transmitted by words and comprehended by the intellect. How could we transmit something vastly more complex even if we knew it -- and I am not claiming I do.
So I would say what you are missing is the direct experience of something higher. Seek and you will find.
Your statement is hard to understand because it claims nothing and is extremely pretentious, of course it gets downvoted.
What you describe is akin to Camus's Absurd, but you chose to fill in the weirdness of existence with some kind of spirituality. That's your choice. Please don't go around claiming to be morally superior to "99.99%" of all humans on earth.
When did I make the claim I am morally superior? I just said most humans are in sleep mode. Even if I said that it's pointless. It's just a belief and not truth.
Also as I said in other comments till you experience a shift none of this makes sense to you. That's okay.
If your mental model does not include some form of making something people want and you are trying to create a startup which has the sole goal of getting people to use and pay for your product, I think you would have a hard time being successful.
Turkey and India is probably not the best examples humanity want to try to emulate when it comes to human rights. And this is coming from someone born and raised in India.
This is going to surprise you, but having more money enables you to spend more of your finite lifespan working on your own initiatives/goals in life (whether they are financially rewarding or not) without needing to persuade someone wealthier than you that they want what you want and are willing to pay to do it.
Ah well I guess I don't understand the relevance of that. Even if you for example have infinite time, doesn't it still make sense a person might be motivated to attain wealth in order to give themselves more freedom in how they spend their life w/o needing to consider affording the survival needs?
Sorry. I was talking about this example where the person choose to risk a financial loss in favor of performing a more virtuous act for the society. He prefered to live a life with integrity over money.
It's cause Countries are imaginary entities and humans just want drama and don't care to get it from their own imaginary entity or some other imaginary entity.
