I think the default is "files-off-disk" image serving (hence NFS, etc). This can be replaced by S3 storage, with the URLs remaining the same (or maybe redirected, if you don't want to pay the bandwidth bill twice).

Disclaimer: I have this setup using Humanmade/S3-uploads for WordPress, but am not currently running Mastodon.

I suspect distributed NFS won't help here when the problem is that a server gets slow / overloaded. In particular, this setup was actually I/O bound and there wasn't more I/O to be had.

> In particular, this setup was actually I/O bound and there wasn't more I/O to be had.

No more I/O to be had from that particular NFS server. If your mounts are distributed over 4+ servers, then you have potentially 4x the available operations available.

Pivotal is also on the steering committee.

Representation is based on contributions to the project, and Google has publicly stated that they look forward to not being a majority on the steering committee (when other companies have exceeded Google's contributions).

There are CNCF projects whose governance is entirely within a single company. I get that there's a great narrative here that "Google doesn't share", but it seems like Google is being held to a higher bar here.

Hi, Knative team member here.

What you're describing with injecting outbound proxies is definitely something that Istio does in service mesh mode. When Knative launched 0.1 over a year ago, we had a pretty hard dependency on Istio. We heard a lot of feedback from users who weren't ready to make that much commitment, and as of 0.3 (last December or so), Knative supports multiple pluggable HTTP routers, including Gloo, Ambassador and Kourier.

If you aren't using Istio in mesh mode, Knative only controls inbound HTTP networking to your container instances, which it uses to implement scale-from-zero, concurrency-based-balancing, and percentage based version rollout. Outbound network is completely unaffected, except that containers may be shut down when they are no longer handing a request.

It depends on the language you're using and how you manage your dependencies.

If you're using Java, see this writeup for how different frameworks can affect startup time: https://medium.com/google-cloud/java-frameworks-performances...

Similarly, if you're using Node or Python, you might want to see if any of your dependencies are enormous with lots of files and slow startup time -- you can check this locally by timing how long it takes to get to the initial listen() call and just print that wall time as you adjust dependencies.

If you're building Golang and you're seeing slow cold starts... I have no idea how you're doing that. For development, a lot of us on the open source http://knative.dev/ side are using Go http servers that take tens of milliseconds to start up, so there's probably some other initialization that's slowing you down.

App Engine is still GA and is still having features added.

Disclaimer: I work on both Cloud Run and App Engine at Google.

Cloud Run is a perfectly reasonable choice for hosting websites -- it's a serverless HTTP platform that uses containers as the base packaging and runtime infrastructure.

Unlike something like Fargate, it supports automatic scaling of containers based on requests, so it will run zero containers if you get no traffic, and 100 containers if you get (for example) 1500 requests per sec. The fully-managed version has a pay-per-100ms of execution model, while the GKE-hosted version uses an existing GKE cluster you provide.

It's stateless, which won't allow many user authentication/CRUD workflows, although recently they added integration with Cloud SQL which is interesting.

Don't think you could use Django on Cloud Run without issues, though, particularly with how it handles Sessions.

I disagree.

If you're doing any modern architecture with microservices using containers, you're primarily doing stateless things (even if it's your web frontend) and pushing the state off to somewhere like Redis/memcached/database.

You basically implied that web frontends don't run in a load-balanced multi-replica set up, which is not true.

Similarly from what you said one might think people don't deploy web frontends to Kubernetes (where containers come and go all the time as they're ephemeral, due to events like crashes, autoscaling), which is also not true.

If you’re writing anything that scales (i.e. has multiple replicas), then you actually store any significant state wrt logins/sessions on your app and you push it out to an external storage. Most web frameworks offer libraries or middleware that let you persist this "state" in external storage.

As a Cloud Run user myself, I suggest adding examples for that use case explicitly in the Cloud Run docs, as those workflows for Cloud Run specifically are harder to mental-model than say Kubernetes orchestration.

It's stateless, but it can connect to stateful services just fine to power things like authentication/CRUD. If the stateful services are also serverless, you can get a completely scalable, stateful system that scales to zero.

We use it in conjunction with Google PubSub and Cloud Storage to evaluate ML models in production and are really happy with it.

Hmm, that might be crazy enough to work.

actually you can run django on cloud run without issues. cloud run is basically gvisor+docker (which can be run on GKE or on google managed servers) its basically built with knative.. btw. even the new appengine can run django. (you can't run background stuff or you shouldn't)

Hi, thanks for the clarification, I'll give it a try then!

Serverless means that you as a developer don't have to manage servers. There's still real computers behind the "serverless".

Knative calls the role of the person providing the servers are "operator". Kubernetes is great for operators because it has a lot of common low-level primitives, and lots of choice if you want to pay someone else to be your operator.

Knative aims to give a similar great experience to developers if you can convince your operator to install it on top of the kubernetes they already have. In particular, if an operator charges you for container runtime minutes on knative, you're getting close to the pay-per use model of lambda or app engine. Also, as you noted, developers should have fewer concepts they need to grok in order to deploy a knative app compared with kubernetes.

I think Knative (and the scale to zero in particular) are best suited for request/response or event-delivery workloads. So you wouldn't want to run (for example) memcache or mysql under knative. FaaS and (HTTP application) PaaS are both good matches.

I think it's an open question whether a PaaS like Google Dataflow is a good match or not. It will certainly require more planning, but I think it's doable.

Yeah but don't forget this is a higher abstraction on top of microservices. I think the big take away here is that if your app falls into the patten of 80% of modern apps (can't remember where is saw that number, don't quote me :) and is split properly you can have all your 'ops' part as configuration.

One of the items that's not present at the moment (and dewitt can probably provide additional color) is the top-level developer shell which puts the pieces together with a minimal amount of client-side work. If you can see Oren's GCP Next talk (https://cloud.withgoogle.com/next18/sf/sessions/session/2204... -- which doesn't show build or events yet), you can see a deployment using `gcloud serverless` to both containers on GCF and a Knative cluster, which differ only in a single flag to gcloud to tell it which endpoint to use.

The short summary if you had a nice client shell would be:

1) Run a command to deploy. That command:

a) determines what build templates are available on your cluster and what language/tools you're using, and finds a match between the two.

b) Creates a YAML definition of your application on the Knative cluster(and stages your source if needed).

2) On the server side:

a) The build component will (optionally) trigger to take staged source and convert it to a container.

b) The serving component will create Istio routes and various pieces to schedule your app into a k8s Deployment.

   i) This Deployment will scale to zero if there's no activity, and scale back up if needed.

   ii) Scale-to-zero is accomplished via a (shared) "actuator" which stalls the incoming HTTP request until a Pod is live.

 c) Additionally, the serving components loads various observability tools like Prometheus and ELK (by default, the no-mon or lite installs skip this) so that you can see what's happening even as your pods appear and disappear.