Not that big data is dead, more like real time data is coming to life, but you need the old stuff around to make a buck or two… Well, that my view. LLMs are transformer model technique are making data more relevant than ever. If you are a business, well you are in for a “now real” digital transformation.
Making data the centerpiece of your business business could mean that your effectiveness of business process could increase several order of magnitudes. Funny thing is, you will not use some else’s model, unless you are building a ChatBox to infer, but you will need to build your own model and be trained in your own business process to be successful.
Consider a bank, here is my prediction of expected outcomes:
Enhanced Customer Experience: The system can act as a virtual banking assistant, providing customers with instant access to their account information, real-time transactions, and balance updates. The system can also answer customer inquiries and provide relevant information, improving the overall customer experience.
Improved Fraud Detection: The system can monitor the bank's financial transactions in real-time and identify any potential fraud, helping the bank reduce its exposure to financial losses.
Automated Loan Processing: The system can analyze loan applications, credit scores, and other relevant data to approve or reject loan applications in real-time, reducing the time and effort required for manual loan processing.
Personalized Marketing: The system can analyze customer behavior, transaction history, and demographic information to provide personalized marketing and cross-selling opportunities, increasing the bank's revenue and customer loyalty.
Real-Time Insights: The system can provide real-time insights into the bank's financial performance, customer behavior, and market trends, enabling the bank to make informed decisions and respond to market changes quickly.
What is interesting to me is, this is just the beginning of what could be…
Yeah, I've noticed more applications just need to focus on making sense of raw information really quickly, but usually don't need an archive to make decisions.
There are lots of interesting things that can happen with "big streaming" than necessarily "big data". Like, cybersecurity is evolving to monitoring and reacting what everyone's machine is doing in the last 15 minutes, instead of having a huge database of hashes you trust. But not a ton of things really utilize what happened, say, 10 years ago on people's machines.
There's definitely some things that can use massive archives of old data, but I have found far, far fewer things that would benefit from it, and often that comes with some very big maintenance hassles. Most of the time, you can just set data retention to 30 days and be done.
They've been working to implement your ideas for decades and none of it requires LLMs or any machine learning techniques. Basic old ETL is more than sufficient.
The issue is that (a) the calculations they need to perform are complex and take time to run (b) there are financial regulations that weave its way through those system and (c) there is a lot of legacy code especially in the core ledger system which "just works" and people are reluctant to touch.
That said depending on your bank you can get real-time account activity, loan approvals in < 5 minutes etc.
Well, that is an understatement. I do agree with you that banks have been trying to fix decades old application.
But in this process, you don’t need ETL, nor all the process and development to accomplish these ideas. Conceptually the idea builds its self (it learns) how to threat the data, quite revealing and near real time. Considering you account for security and privacy, then you basically shift your input into the data stream and using a natural language get the data output you need, not clunky apps.
Imagine I just login, and say:
me>how much do I have?
bank>You have 100$
me>Please send 50$ to 1003
bank> Are you sure? Please add your security code to confirm
bla bla…
All this with little intervention.
Banks spend hundreds of man hours developing a lacking application while delivering a very poor customer experience. They spend millions on running decades old applications because it so expensive to change them… and thus the circle continues…
I’m really exited to see DataBases disappear conceptualy, data entry, mostly all that just disappear… I will ask my ChapBot for statement, give me a personal investment advice, and classify all my purchases and see where my wife has been spending all my money, all from the confort of my phone.
it’s a brave new wold we are wakening up to, that to me is exciting. And coming from having helped several major banks build their infrastructure, it’s just a boost to talk about something fresh, no more Hypervisor, core count, db licenses, ect. Ok, I’ll concede it’s pretty much the same old, just the nemonics will be different… How many GPUs, how quickly can you spin a container, how fast if your S3 datastore… oh wait, there is that circle again… >:D
Making data the centerpiece of your business business could mean that your effectiveness of business process could increase several order of magnitudes. Funny thing is, you will not use some else’s model, unless you are building a ChatBox to infer, but you will need to build your own model and be trained in your own business process to be successful.
Consider a bank, here is my prediction of expected outcomes:
Enhanced Customer Experience: The system can act as a virtual banking assistant, providing customers with instant access to their account information, real-time transactions, and balance updates. The system can also answer customer inquiries and provide relevant information, improving the overall customer experience. Improved Fraud Detection: The system can monitor the bank's financial transactions in real-time and identify any potential fraud, helping the bank reduce its exposure to financial losses.
Automated Loan Processing: The system can analyze loan applications, credit scores, and other relevant data to approve or reject loan applications in real-time, reducing the time and effort required for manual loan processing. Personalized Marketing: The system can analyze customer behavior, transaction history, and demographic information to provide personalized marketing and cross-selling opportunities, increasing the bank's revenue and customer loyalty.
Real-Time Insights: The system can provide real-time insights into the bank's financial performance, customer behavior, and market trends, enabling the bank to make informed decisions and respond to market changes quickly.
What is interesting to me is, this is just the beginning of what could be…