We use HCL broadly across the HashiCorp portfolio. It was popularized by Terraform, but also used in Vault, Consul, Nomad, and Packer.
Important to note, that HCL is designed to be completely interoperable with JSON. In practice, we find that HCL tends to be more human readable and writable and is significantly less verbose.
However, if you are doing any sort of machine generation, it can be convenient to generate JSON and feed that in.
We've continued to grow the Nomad team, and are working towards the big 1.0 release milestone later this month. The OSS usage continues to grow double digit every quarter and our commercial offering generates millions in revenue. Nomad is also the backbone of both HashiCorp Cloud Platform and Terraform Cloud. Suffice to say, we continue to support Nomad and depend on it!
That's great to hear. I want to thank you for your work on Nomad. Its been a pleasure to use and has allowed us to scale without unnecessary complexity.
We don't have a white paper on this yet, but we have a white board video that explains both how it works conceptually as well as at a more technical level of deployment architecture and data flow. https://www.youtube.com/watch?v=tUMe7EsXYBQ&feature=emb_titl...
Armon, just wanted to say your whiteboard videos are excellent. And the clarity of thought demonstrated in them over the years has been a great ad for the products too. The low tech aspect also feels more human.
But I had a chuckle at the idea of you wheeling a whiteboard into your house (if that is where it is filmed).
So does it intercept all connections on that port (from the client app) and pass them along? Or do I need to reconfigure my client application to talk to localhost:whatever? Your only example is that curl using a hostname, it's not really clear.
mostly copy-pasta from an earlier comment[0] of mine:
https://github.com/99designs/aws-vault/issues/578 was for an issue with remote servers accessing the localhost ec2 metadata service that aws-vault can run, that worked exactly by using DNS rebinding. It was fixed only months ago, so it seems like this is a developing area and if I were on a red team or pen testing, I would play around with more.
I visualize the "localhost hole" problem of blindly trusting localhost as an air gap in a pipe (like [0]); anybody could come along and either drop poison in the pipe, or redirect the water coming from the top to their own bucket, or both.
I appreciate that Boundary gives completely generic identity-aware-authenticated TCP sockets, but I don't know of a way, today, to make those not accessible to browsers through dns rebinding attacks.
This is probably much much too far in the weeds and this is unlikely to contribute to a major breach (unlike the aws-vault one where of course attackers would try to access the fake metadata service on the default port, because it's high-value and on a well-known port), but I'm interested in the space.
Glad you found Consul easy to get going with. We definitely focus on enabling K8S as a first class platform. There is documentation on setting up Consul and operating with Kubernetes here: https://www.consul.io/docs/platform/k8s/index.html
HashiCorp builds DevOps tools for modern applications. We build open source tools including Vagrant, Packer, Terraform, Consul, Vault, and Nomad. We also have a set of cloud services (Vagrant Cloud, Terraform Registry, Terraform Cloud, etc) that are high scale SaaS services. The HashiCorp products are downloaded tens of millions of times per year and power tens of thousands of organizations. Join our growing team and help organizations deliver modern applications using DevOps practices and enable dynamic multi-cloud infrastructure.
At HashiCorp, we are committed to hiring and cultivating a diverse team. If you are on the fence about whether you meet our requirements, please apply anyway!
HashiCorp builds DevOps tools for modern applications. We build open source tools including Vagrant, Packer, Terraform, Consul, Vault, and Nomad. We also have a set of cloud services (Vagrant Cloud, Terraform Registry, Terraform Cloud, etc) that are high scale SaaS services. The HashiCorp products are downloaded tens of millions of times per year and power tens of thousands of organizations. Join our growing team and help organizations deliver modern applications using DevOps practices and enable dynamic multi-cloud infrastructure.
At HashiCorp, we are committed to hiring and cultivating a diverse team. If you are on the fence about whether you meet our requirements, please apply anyway!
Radix trees are one of my favorite data structures, and widely under used. For many "dictionary" type lookups, they can be faster and more efficient than hash tables. While hash tables are commonly described as being O(1) for lookup, this ignores the need to first hash the input, which is typically an O(K) operation, where K is the length of the input string. Radix trees do lookups in O(K), without needing to first hash and have much better cache locality. They also preserve ordering allowing you to do ordered scans, get min/max values, scan by shared prefix, and more.
If you combine them with an immutable approach (such as https://github.com/hashicorp/go-immutable-radix), you can support more advanced concurrency, such as lock free reads. This is important for highly concurrent systems to allow scalable reads against shared memory.
To keep it brief, we are more committed to Nomad today than before. The team has doubled in the last year, and we plan to grow further next year as well. Our goal has always been to build a simple, general purpose scheduler, that composes well with the rest of the HashiCorp ecosystem.
Kubernetes is an important ecosystem and a platform we tightly integrate with across our other products (Terraform, Consul, Vault). We've always believed that our tools would be "mixed and matched" with different technologies, and that pragmatically we should support the broadest range of integrations.
Nomad is an important piece of our ecosystem, and we have many open source users, enterprise customers, and our SaaS offerings are built on it. Rest assured, it's not going anywhere!
That is exactly right. At the bottom of the blog post we touch on this, but if you are using consul-template to provide secrets via a configuration file, it can either restart or reload (signal) the application to pickup the changes. Alternatively, an application could be Vault aware and use the SDK programmatically.
Important to note, that HCL is designed to be completely interoperable with JSON. In practice, we find that HCL tends to be more human readable and writable and is significantly less verbose.
However, if you are doing any sort of machine generation, it can be convenient to generate JSON and feed that in.