All excellent points. Right now, Neptyne works much like a Google doc in that it's private to you but is easily sharable to pretty much anyone. If our users are looking for it, a self-hosted option is certainly something we'll explore.
Then there's things like PII protections as you said, data provenance, that you could imagine baked into the spreadsheet directly. We haven't gone in that direction but I think it's interesting to consider.
What sort of features around data security would you hope to see in a spreadsheet?
Thanks! We've talked to a lot of people in hedge funds and investments, and you're right that a lot of them care about a self-hosted solution. This is definitely on our radar as a potential future path for the product.
It is true that the people who are most productive today in Neptyne are those who are pretty familiar with Python. Part of our long-term goal is to make this kind of programming more accessible to beginners, and we talk to a lot of people who wouldn't call themselves "programmers" but are familiar with spreadsheets. Our hypothesis is that if you give these people a frictionless way to incorporate programming into their workflows, they'll learn quickly and reap the benefits.
That does sound like a fitting use case. Technically speaking it should be OK with 70k rows, but we've got some optimizations that might need to go out before it performs well. Ideally, you should be limited only by the memory allocated to the Python kernel, but today there are some other limitations that get in the way.
All great tools to be sure! I am a huge fan of numpy and pandas especially. For plotting in Neptyne I usually opt for plotly over matplotlib/seaborn. Agreed too on the shell -- that's why we built one into our product.
I am curious though, do you typically share/collaborate on these spreadsheets with others? Do you have a neat way of packaging up the spreadsheet+Python code?
We do support live collaborative editing, though it isn't all built out yet. If two people are working on the same tyne, you'll see the presence of other collaborators and you'll see their changes reflected live, but the code panel updates with last-writer-wins at the moment. We're working on making that better.
The main technical difference between us and something like Equals is that we give you a full Python environment (in the form of a Jupyter kernel running in a Docker container) for maximum programmability and flexibility. This means that you, as a Neptyne user, can connect to virtually any data source by importing the appropriate package and setting credentials. And of course you can use Python in the actual functioning of your spreadsheet, not just as a one-way data import step.
That's probably something we should dig more into in the docs, yeah. Neptyne works like a ipython/Jupyter notebook in that everything is in scope all the time. So you can write your code in an imperative way (e.g. `B10 = some_value`) within a function, or you can do use a functional style.
We handle spreadsheet-style reactivity by building the dependency graph out of any cell addresses mentioned in the cell itself. So if you have `=my_func(A1)` in one cell, it will re-run when A1 changes. But if my_func reads from, say, B1, it will not automatically re-run when B1 changes.
It's a conscious decision. The spreadsheet recalculates itself, the python code doesn't. Recalculations in the spreadsheet are determined by the graph (cell Q3 depends on cell D4 because it mentions it in it's formula like "=D4 * 3") If Q3 calls a python function ("=my_function()") and in that function D4 is mentioned we don't add it to the graph. It is already hard enough as is to see the dependency graph in a spreadsheet - adding implicit relations to it would make this almost impossible.
In Observable, a cell can define a function or return a value. When the value of a free variable within a function definition changes the function is redefined rather than the function being rerun. So there is a clear difference between points at which the function's bound variables come into play (function invocation) and the points when changes to the free variables have an effect (function definition). It's far from unmanageable in that "reactive notebook" setting. Maybe spreadsheets are the problem...
