If you want to play with Pyodide in a web notebook you can try Starboard [1][2].
A sibling comment introduces JupyterLite and Brython, which are Jupyter-but-in-the-browser, whereas with Starboard I'm trying to create what Jupyter would have been if it were designed for the browser first.
As it's all static and in-browser, you can embed a notebook (or multiple) in a blog post, for instance to power interactive examples. The bundle size is a lot smaller than JupyerLite for the initial load - it's more geared towards fitting into existing websites than being a complete IDE like JupyterLab.
I’m concerned with the performance. It’s up to 16x slower than native Python, which we already know is 100x(?) slower than native code.
So roughly 1600x more clock cycles are consumed than is necessary.
I’m already jaded with the extreme bloat of JS frameworks as it is (how many times does my computer need to ramp up fans because of your React blog?). This could potentially be an order of magnitude worse.
I think the main usecase for Python in the browser using Pyodide is to enable scientific computing and visualization in the browser, and also as a way to ship small Python applications to those that don't have Python installed (e.g. drag and drop a CSV file which gets processed with pandas). Lastly it's useful to power interactive coding environments without needing to spawn a pod per user (and fighting off people using them to mine bitcoin).
Building a website or SPA entirely backed by Python seems like the wrong use of this technology. The same goes for crunching large amounts of data.
The quality of an answer is often orders of magnitude more important than the speed at which it's generated. Speed is important, but it is always secondary.
You wouldn't pull this in for a snappy, reactive UI driven by Python. You pull this in so that you can re-use a server-side module to perform some activity that isn't critical and so not worth re-writing in JavaScript. In another 5 years (assuming Python is still of interest and that this solution worked), it will have been refined and optimized where maybe you do reach for it by default.
Genuine question: What are the quality improvements offered by Python? Is it mostly access to the scientific community's Numpy/Scipy/Matplotlib? The language's semantics?
Most ML papers are implemented in Python, so you need it in order to be able to play with the latest and greatest toys, or you can wait for someone to port them to your favorite language, if ever.
And I would say that this is because of the language's syntax & semantics. Pythonic code looks like pseudocode, aka business logic.
There is very little boilerplate, and libraries are designed to hide what little there is.
So this means that mathematicians who hate verbosity can read and write their algorithms with little cognitive overhead.
Then you start to get the network effect where libraries are implemented or wrapped in Python, and people use those and start adding abstractions etc. The two main deep learning libraries (TensorFlow and PyTorch) are there, and those have huge gravity wells.
> performance since the 2019 announcement has improved greatly: "Performance ranges between near native to up to 3 to 5 times slower, depending on the benchmark."
Concrete Python applications wouldn't be 100x faster if re-coded in some low level language.
In most places where performance matter Python packages would be using either C extension, Cython or numba to get near native performance. Pyodide is able to build those packages (except for numba). So overall it's currently 3 to 5 slower than native Python (which uses C extensions). See detailed benchmarks in https://hacks.mozilla.org/2021/04/pyodide-spin-out-and-0-17-...
Yes, installing pure Python packages that have wheels from PyPi works. Assuming they don't use functionality that's not supported by the WebAssembly VM such as multiprocessing, threading (for now), and sockets.
I use this for the isort in browser demo: https://pycqa.github.io/isort/docs/quick_start/0.-try/ it was really awesome to be able to directly use a Python package without any wrangling or modification as is usually required with other Python on the browser solutions.
Has anyone gotten this to run in a wasm runtime on the server? At some point I saw that wasmer had a fork adding server support, but it was out of date. Main challenge seems to be assumptions about running in the browser?
My motivation is that I’d like to run sandboxed Python scripts on the server, in a wasm runtime embedded in C.
How does Node play into this? Can I embed a v8 engine in my C program and use pyodide, or do I need to bring Node along too? Does the compatibility layer depend on any Node-specific concepts like the event loop or garbage collector? (I think maybe V8 does its own GC? This just shows how thin my experience is with this topic.)
Neat:
"The 0.17 release has a number of interesting features. Support for Python asyncio has been added, so that Python coroutines can run in the browser event loop; a JavaScript Promise can be awaited in Python and vice versa with Python awaitables. Error handling has also been upgraded so that exceptions generated by Python can be caught in JavaScript; that can be done in the other direction, as well."
I tried brython once and wrotesome video/image gallery with it, I was quite happy, but it ended up being quite slow.
I'm super excited about pyodide, although:
* I can't find a discord or IRC channel to ask questions, I guess the project is still young. Last time I've seen this project, it looked like a scientific software suite, which came with a lot of stuff I don't need.
* I wish there was more examples or samples, especially with events and the DOM.
* I've glanced at the doc and howtos, and it seems it requires a server, which I don't understand. Brython seems much more straightforward.
* I've read that memory leaks can happen in multiple places
* This example seems like misleading advertisement, in my view, at least for now:
from js import document
x = document.getElementById("myElement")
(Or I might need to read up more doc to understand how to get to that point)
Well you need to be able to serve static files. So if you are building locally you need to start a webserver, otherwise you can use the JsDelivr CDN.
Memory leaks can happen if you translate objects between Python and JS. So of it is unavoidable because it's difficult to know when to destroy Python objects from JS vice versa, but lots of work on it has been done in the 0.17 release with more to come in 0.18
> Well you need to be able to serve static files. So if you are building locally you need to start a webserver, otherwise you can use the JsDelivr CDN.
I don't understand, can't I just use offline, static files ?
I would still expect to use my own webserver, unrelated to pyodine, like flask or whatever.
> Yes, we are working on adding more examples.
Glad to hear this :) !
> Why do you find the example misleading?
I haven't really seen that example in the doc or elsewhere, and I don't understand the steps required to make such example work offline or without a webserver.
EDIT:
Another cool thing of brython is that could "inline" a python script inside a html file, such as:
If you just open an HTML file in your browser via its local path (e.g. /home/user/...) the browser will prevent you from loading other files by JS due to CORS. So we need a web-server to set the appropriate CORS headers. Any web-server would do.
> Running the Python interpreter inside a Javascript virtual machine adds a performance penalty, but that penalty turns out to be surprisingly small — in our benchmarks, around 1x-12x slower than native on Firefox and 1x-16x slower on Chrome. Experience shows that this is very usable for interactive exploration.
That data is from 2 years ago. Lower down in the post it has an update for the present:
> performance since the 2019 announcement has improved greatly: "Performance ranges between near native to up to 3 to 5 times slower, depending on the benchmark."
Python's object semantics are extremely different from JS. There might be _some_ ways of emulating them with JS thanks to Proxy objects nowadays, but you're looking at a pretty big uphill battle I think.
That's an interesting way to put it. Python does have a form of prototypical inheritance (given how objects end up being `__dict__`s with behaviour defined through `__class__`, though not only that!).
Python descriptors are extremely flexible though, and I _guess_ you could emulate almost all of it through Javascript proxies, but I can't imagine the performance of that being great. Hash tables would also not be re-usable between JS and Python (since Python hash tables allow many more varieties of objects), and Python has a distinction between attributes and keyed access (which JS lacks).
Python exposes much more of the global environment and internals than JS does, and doesn't have some of the requirements JS has put on itself for security reasons. I'm not super sure you could emulate Python blocking style with a JS transformation statically. Blocking Python for something that is only available in a non-blocking JS environment would require effectively wrapping every Python operation (since almost every Python operation can have arbitrary side-effects).
I think there are people who write sort of syntax-y transpilers, but that's not really going to lead you down to nice code re-use.
Numbers. Everything is a float in JS. Python has ints that transparently turn into bigints on demand, floats, and if you need it a whole library of C numeric types to do interop (along with complex numbers and all kinds of other great stuff in python's standard library).
My point was more about the object model itself being incredibly similar, rather than the data types.
However, much appreciated.
It's perhaps interesting to note that browsers pretty much already implement this behaviour for js, ie., by spec everything is a float -- but vms don't start with a float type for much of the numeric data stored; and transparently move to it when needed.
- you can replace builtins in python.
- you have __dunder__ methods and multiple inheritance.
- JS has an ever running event loop and is async by default.
- python is more strickly typed.
It's not impossible (see brython), but it's not easy either.
I don't think you have that quite right. Objects don't inherit from other objects in Python — you can't do `class Foo(object())`, for example. Python has class-based inheritance.
"If you want to inherit from an object, just create a class based on that object" is a description of how you'd imitate prototype inheritance in a language that only has class-based inheritance. It does not mean you're actually looking at a prototype-based language, and in fact strongly suggests the opposite. You can't get inheritance in Python without engaging with the class system.
I suppose you could just say that classes are a particularly restricted sort of prototypes, and thus any class-based language is effectively prototype-based. But at that point "<OOP language> has prototype inheritance" is just tautological based on the set of definitions you've chosen.
It's not tautological. I don't think C#, C++, Java, etc. can have classes proxy for objects. Classes there aren't objects. You cannot write "return NewClass".
Python is prototypal in just the same way js is: inheritance is a daisy-chain of references between objects.
JS is "maximally prototypal" if you like, where any object may be refernced. In python, only objects of type "type": but that isn't a big restriction, as shown.
python has a lot of features. But people use a small subset of it. So instead of supporting all of python, why not stick to the small subset that people actually want to use to solve a problem?
Different Python programs use different subsets, so as you work towards supporting more programs the union of the subsets becomes essentially the full set, I would assume.
The distribution of popularity of the features is not uniform. My intention is to carefully look at the intersection of relatively less used features and features that make python slow/inefficient and exclude them from the supported subset.
Because a large part the Python ecosystem uses packages (and stdlib modules) with C extensions and you can't easily transpile those. The end goal of Pyodide is to be able run existing projects mostly without modification in the browser, and that is not possible if one doesn't support Python C extensions.
The parent is talking about transpiling python to JS. I don't see why people who hate JS would like python. The language itself is not any better IMO (excluding ecosystem here).
edit: following some comments I want to add that the situation is symmetric IMO. I don't see the point in transpiling any of the two to the other.
>>> a = 5
>>> b = 5
>>> a is b
True
>>> a = -4
>>> b = -4
>>> a is b
True
>>> a = 300
>>> b = 300
>>> a is b
False
Every language has these, JS a bit more (because mostly impossible to fix thanks to backward compatibility). I don't understand why you would do that anyway.
Most linters would display errors even if you did (in both languages).
In python it completely makes sense because it's an object oriented language and threat everything as object. In fact it's very consistent about that. Even classes are object and instance of metaclass. Difference between '==' and 'is' is came from that. This is not the case with Javascript. It's sad many peoples who are dealing with python don't understand that.
It is a pitfall. It's not about what python developers think about it. It is about what people from other languages experience when they learn the language.
(Though it's not my main point, I want to add as (also) a python developer, that I think this is a terrible design choice, which could easily be avoided. In this case python doesn't "threat everything as object" it treats small int literals differently than bigger int literals)
Small integeres are object in python. They are cached in some implemention. If you only care about shallow syntax, yes it's confusing. But if you care more about semantics, it's completely consistent and doesn't make especial case.
Many programmers do care about the "shallow" syntax.
Regardless, if, as you said, the operational semantics of the language is inconsistent among implementations and depends on internal caching, it only makes it worse.
Great. Thank you. I'll remove the "is" operator from the supported subset and perhaps rewrite the commonly used "a is None" to idiomatic code in the target language.
You should always use 'is' with None. == Is operator that can be overloaded and 'is' is object identity comparison. It make sense when you understand object system. You will be terribly messed up when use some numpy array == None.
I really wish it didn't though. I started working with js ~5 years ago and haven't used == a single time, there are eslint rules that error on it[0] and you can reject push to repo if you have them, which many companies do. It's not a problem.
It's just what you are used to. I dislike Python, I hate PHP, but I really like JS. I've seen people like PHP and hate Python too.
You see more JS hate because it's a more common scenario that people happy with PHP/Python/Whatever would be exposed to JS later, than the other way around.
You see a bit more Python hate lately because ML is all the rage and Python is to ML what JS is to the web.
All mentioned languages having some weird backwards compatibility quirks doesn't help either.
I find it good that people are trying to port their favorite languages, although personally I'd stay on the happy path for the target platform. Especially the browser, as it already has weird APIs, JS or not.
I agree, I was fluent in JS and other languages before learning python and hated it for a while. Eventually, you get used to the bad parts of any language so they become mostly transparent (and every language has bad parts).
Specifically, now that I'm very much OK with both python and JS I find transpiling one to the other kind of pointless in terms of any language property (can make sense in terms of ecosystem).
Brython => a JS Python implementation. It will not behave like cPython for somethings, does not have access to the whole stdlib nor the scientific stack. But it is quite small (700ko without their stdlib) and has native bindings to the DOM. Use it if you feel like writing your web page in python (I wouldn't, I love python but 700ko+ with no framework loaded yet is way too much).
Pyodine => the cpython implementation, but compiled to wasm. It's very heavy (10Mo), but it brings the real, whole cpython implementation in the browser so it will behave the same and have the entire stdlib. It also comes with optional NumPy, Matplotlib, pandas, SciPy, and scikit-learn. Use it if you want people to write python in the browser, but don't want to expose your server to their code.
That's totally different. Pyodide is CPython in WebAssembly, and can use existing Python libraries (including those written in or backed by C), like NumPy, Pandas, and MatPlotLib. Brython transpiles Python code to JavaScript, so can handle some pure-Python libraries only: https://stackoverflow.com/questions/57517734/how-to-import-n...
I feel Blazor WASM is a better choice in every way.
To elaborate : Blazor uses C#, is more performant, .NET 6 is bringing AOT for even better performance, has an ecosystem of tools to build sites with, integrated well into the .NET ecosystem, has a great set of libraries to choose from.
The latter two, certainly, are adequate for class exercises and demonstrations. No idea if they are good enough for production of course, which Pyodide seems to be claiming.
A sibling comment introduces JupyterLite and Brython, which are Jupyter-but-in-the-browser, whereas with Starboard I'm trying to create what Jupyter would have been if it were designed for the browser first.
As it's all static and in-browser, you can embed a notebook (or multiple) in a blog post, for instance to power interactive examples. The bundle size is a lot smaller than JupyerLite for the initial load - it's more geared towards fitting into existing websites than being a complete IDE like JupyterLab.
[1] https://github.com/gzuidhof/starboard-notebook
[2] https://starboard.gg