Skip to content

product

  • in product
  • 5 min read

focusing on on-prem in 2024

2024 will be a new chapter for kraud, taking learnings of the past year to focus on what matters most to our customers.

Highlights

  • Our primary focus will shift towards on-prem cloud and data storage.
  • We want to replace the (large) office NAS and have some exciting products in the pipeline.
  • Reliability and Durability will be key criteria. 
  • We're in the process of finally finding a serious company name.

public cloud price increase

Our primary mission is privacy and sustainability. Low prices are a side effect of energy efficiency rather than the goal. With thousands of free tier users, we see a clear demand for a cheap public european cloud. Yet in 2023 we have not been doing a good job of taking that audience onto the sustainability journey. Educating a wide user base for free is a costly investment that directly conflicts with our goal to remain independant of destructive venture capital.

Price sensitive users are encouraged to look at lower price competitors such as Hetzner, or consider AWS free tier. The datacenter in Berlin will actually need to expand due to demand from private cloud customers, so we may soon need to close registration of new free accounts.

It's a long road to a carbon neutral cloud. Focusing on customers with at least 64GB and up allows us to invest into longer term relationships where we can actually help customers grow sustainably and look at their compute needs individually.

Storage First

2023 was also a year of seeing where we actually compete on the market in terms of product quality. Turns out competing data storage solution are terrible! Who knew.

We've upgraded our cloud SAN to a high performance 100G redundant NVME cluster so you can do compute directly on redundant dynamic storage. Thinking this is the norm, but it really is not.

Of course you've got pure-storage, absolutely amazing stuff if you have the budget, but most smaller companies struggle with basic office NAS, zero redundancy, slow transfers, etc. Even larger competing cloud service providers are unable to meet customers storage performance demands. Out of the public clouds, only AWS is able to match our specs. Surprising.

At the same time, customers have made it very clear that they expect more from us in terms of usability. While we currently don't even have a web ui for storage, this will improve dramatically over the next year.

Outposts, the work from home cloud

Making the cloud come to where your data already is. Stored safe and secure in your own premise. 

A clear signal we got from you, our customers, is that you like our product for its ability to work on premise.  Some customers aren't just disliking the cloud for pricing but they simply arent allowed to upload data there. We're very excited about serving that niche and reducing the barrier to entry.

Not only is the on-prem experience going to improve in general, we're also finally adding a new device: the Mac Studio.

We recognized the difficulties in home office with moving data around with poor Internet. Some people log in to their office server to work on things via ssh. Outposts are going to give every one of your team members a part of a unified data lab. Copying files between different outposts and the central cluster is significantly faster than doing it over NFS, samba, web, etc, because the outposts carry metadata and can copy only the relevant changes.

When working with VMs we want to enable moving the entire VM between different outposts. This is already possible with containers today, but we learned that this doesn't fit the market need so we'll give VMs the same capabilities. Just install a VM to your liking and move it back and forth between your outpost inside your virtual lab. 

It will actually keep the same public ip address no matter if it's running at home, in the cloud or in your lab. We might even be able to move it without rebooting.

We're also planing a feature that lets you just take a drive out of your outpost and physically move terrabytes of data between them. Need that 10TB synchronized today? No worries, we'll send someone to pick up your encrypted drive within the same day and sync it with your main cluster.

Virtual Labs, let the work come to the data

Sometimes you want the opposite. There's a large data treasury in the office, but you don't want it to leave the building. Instead you want to bring the analysis to the data. We want to enable this workflow by creating isolated compute spaces that allow anyone to run compute on specific files you share with them. Our technology already enables you to be safely isolate users from each other, but we want to build a proper workflow management that fits the specific requirements of data based research.

A data lab can groups multiple resources, such as a specific amount of memory and CPU power over a whole cluster of machines. It also specifies the exact file volumes that are accessible. 

Volumes from outposts can also be selected and will be available to others in the virtual lab. This enables sharing of "personal" data with a lab, while retaining physical control.  A possibly interesting feature to deal with the legal ambiguity of data "location".

Data clean rooms are using memory encryption and attestation so you can run analysis on someone else's outpost without revealing the content of your own code or input data. A common requirement in the data trading business.

let's hang out more

We have some absolutely amazing customers and partners who are helping us achieve sustainable computing. I will spend much of the year traveling to different venues to meet old and new customers, learning about how we can make your compute usage more effective and sustainable.

Hope you're all having a great start of 2024.

  • in product
  • 2 min read

ipv4 docker overlay networks are now supported

Each tenant on kraud has a least one VPC, a "virtual private cloud" as the industry term happens to be. VPCs are fully isolated networks from other VPCs, and can span across multiple datacenters, including on-prem.

using the ipv6 VPC network

Each pod can connect to every other pod within the same VPC (again, also across data centers) over a wireguard encrypted channel. A VPC is a layer 3 ipv6 transport, which is the most efficient way to do routing between many different sites and participants.

Domain names of pods resolve to their ipv6 vpc adress by default. You can simply start two pods in the same namespace, let's say "db" and "app" and then connect to "db" from the app. This works out of the box with almost all modern software.

You can also connect to an app in a different namespace using its fqn, i.e. "app"."namespace"

using ipv4

Feedback from you, our users, included the feature request to use ipv4 between pods instead of ipv6, because some legacy applications do not support v6. Also some docker compose setups include multiple networks with fixed ip addresses that do not work well with dynamically assigned v6 addresses.

Starting today, every namespace additionally has a default ipv4 overlay assigned. It is implemented using ip6ip on top of vpc, so it's not as efficient as the default vpc routing, but it allows for convenience features such as assigned fixed ipv4 addresses

You will notice that docker --context kraud network ls now returns slightly different output. Since docker cli lacks a large amount of features, we need to condense the information a bit

1
2
3
$ docker --context kraud.aep network ls
NETWORK ID     NAME                      DRIVER    SCOPE
v1223123123    nextcloud/default         ipip      global

the first part is the namespace, the same thing returned by kra ns ls

1
2
3
$ kra ns ls
NAME             CREATED              
nextcloud        2023-09-11 12:16:30

the second part is the overlay network

1
2
3
$ kra overlay ls
aid           namespace        name     driver  net4             net6                   
v1223123123   nextcloud        default  ipip    10.198.62.0/23   fd42:1a8:0:2a09::/64

as you can see, this one has an ipv4 assigned. you can specify which one!

docker-compose.yaml
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
networks:
  v:
    driver: ipip
    ipam:
      config:
        - subnet: 192.168.0.0/16
services:
  k2:
    ...
    networks:
      v:
        ipv4_address: 192.168.2.0/24

hopefully this brings us closer to direct docker compose compat, and helps you with legacy applications. We're always happy to hear about fedback and feature requests. If you need help, dont hestiate to contact [email protected]

  • in product
  • 2 min read

Full Kubernetes is now generally available

get your very own real k8s for free with kraud.cloud

Good news, kubernetes is now available for everyone. k8s app

There's a free tier version with a single node and v6 addresses, as well as a version that has 3 nodes and uses v4 addresses. As you probably know, v4 addresses are rare so they're not available in free tier.

Deploying k8s on top of kraud enables you access to the full original k8s experience, including the ability to install CNI plugins and custom resource definitions.

You can also download and change the compose spec and launch k8s with kra

kubectl and compose are incompatible use cases

Thanks to the helpful feedback of our users we have learned that the original idea of emulating the k8s control plane is not the experience you're looking for.

The kraud k8s control plane is not fully compatible with the k8s ecosystem, and will always lack behind k8s upstream. With the availability of real k8s as app, we have decided to completely stop working on kubectl compatibility for the kraud control plane. kubectl may function for a while until we completely remove the api.

We will instead further enhance docker compose compat. For example just recently we introduced the ability to use overlay networks, which k8s does not support.

Going forward there are 4 ways to use kraud:

  1. launch or publish a complete app on the marketplace
  2. launch k8s from the marketplace and work with the k8s ecosystem
  3. launch your own app on managed docker compose clusters with kra
  4. launch a bare linux VM and use legacy tooling

as always we love your feedback and wish you an efficient computing experience.

  • in product
  • 2 min read

NFS is production ready | LV2 will have lower guarantees

This year we introduced NFS for highly available file volumes and LV2 for very fast local block storage.

NFS is worry-free, just use NFS

NFS is sufficiently complete that its the second service to be marked production ready in the product specification and the first to come with a money back guarantee. It is fully single-zone redundant with active/passive failover and automatic snapshots. This is the default storage everyone should be using. It offers complete worry free file storage that works with most docker containers out of the box. NFS is also accessible via webdav and S3. This allows easy collaboration on file based workloads.

Later this year we're planning to make snapshots explicitly controlled by users, so that you can instantly rollback yourself instead of asking customer support to do it.

LV2 requires special attention

Some customers prefer "virtual disks" instead of file storage. The primary advantage is that they're generally faster for single process workloads. There are several different ways to implement "virtual disks". If a competitor does not tell you which technology they use, it is very likely what we call LV1, that is locally redundant (RAID) storage directly attached to the host. In case of a host failure, the volume becomes unavailable and the associated pod will not be rescheduled.

With LV2 we tried to approach similar high availability and guarantees to our HA system, but ultimately the two worlds are just too dissimilar, and customers have much lower expectations. Instead we are reducing high availability of LV2 for the benefit of much quicker launch times. Users may choose to either restore from cold storage with a Recovery Point Objective of 1 Hour, or wait for manual recovery of the host (usually within a business day).

If you look out there you may notice that services like AWS EBS or Hetzners CEPH volumes offer much lower RPO than krauds LV2. This is because they're network volumes, not directly attached. These products are very unpopular, so we currently do not offer them. The primary disadvantage is that network storage has a significant latency penalty, at which point you might just use NFS.

However, if you would like to see zero RPO block volumes at kraud, let us know!

  • in product
  • 3 min read

get ready to move your pods to Berlin

Backstory

Earlier this year we semi-announced the intention to move to a new datacenter location. Kraud is on a mission to build carbon negative infrastructure, and the current datacenter provider misrepresented the origin of their elecricity in a way that does not align with our values.

The new location is 3U Telecom in Berlin. 3U Holding, the parent company of 3U Telecom is heavily invested in renewable energy, and we hope to have found a partner that shares our vision.

We also picked new network partners, inter.link , Core-Backbone, and BCIX, significantly increasing the connection quality of the new site.

We spent half a year building out the new site with an entirely new storage architecture based on ZFS. The old site had extremly poor IOPS and frequent total cluster freezes due to bugs in ceph. We can now offer LV2 storage with half a million IOPS, and automatic instant snapshots. A huge shift away from thinking first about compute to thinking first about storage.

The old site will be shut in July, so please prepare your workloads asap.

TLDR

Moving pods with no storage attached is easy. Just delete the container and relaunch with the zone label -l kr.zone=uca. The site will become default in july, after which you need to specify -l kr.zone=yca to launch a pod in the legacy zone.

If you have storage attached, moving is slighly more involved. We currently recommend moving from rbd to lv2 and from gfs to nfs, as this will most likely match your expectations.

  1. stop all pods
  2. create a new volume with docker volume create myvol --driver nfs
  3. copy all the files from the old to the new volume, for example by using webdav or with docker exec, depending on your use case.
  4. delete the old volume with docker volume rm oldvol
  5. recreate your pods with -l kr.zone=yca

if you need assistance, don't hessitate to ask on discord or send an email to [email protected]

Berlin is a huge step forward

with the new 100G infiniband SAN, we're starting a new chapter of thinking about storage as a primary feature. Data is critical to all apps, and we want to make sure to offer a great experience in that regard too.

Being connected directly to an IXP also means we're now an actual proper datacenter with redundant paths to the internet and high speed direct peering to a significant chunk of the European internet.

Finally, i personally live in Berlin, which of course had a huge impact on the choice of locations. Having direct access to the machines enables me to run new and exciting new hardware features like GPU and risc-v.

If you're a free tier user or paid customer, i thank you for being on board this adventure. I hope you're happy with our new site. Let me know if you have comments or feedback.

picture of berlin server rack

Introducing the kraud cli: kra

From the beginning of the project we always strived for compatbility with your existing tools, be it docker or kubectl. Your feedback is always greatly apprechiated, as it helps us clarify what that exactly means in practice. How much compat is good, and where do the existing tools not work?

We haven't reached a stage where this is entirely clear yet, but the trend is pointing towards

  • Fully supporting the docker cli
  • Building a custom cli to supplement docker
  • Freezing kubectl at 1.24
  • Partially supporting the most popular of the many incompatible docker compose variants

Particularly kubectl is a difficult choice. Kubernetes is a standard. But unfortunately, it's not actually a standard, and keeping up with upstream does not seem feasible at the moment.

Instead we will shift focus entirely on supporting docker and docker compose. The compose spec is weird, and inconsistent, but it is simple and hence very popular. Most of the confusion we've seen in practice is easily addressable with better tooling.

So we are introducing: kra

The kra commandline program works on docker-compose files and will implement some of the processes that docker does not do at all (ingress config currently requires kubectl) or does incorrectly (local file mounts).

Specifically a pain point in some user setups has been CI. Since we don't support docker build yet, users build on the ci machine and then use docker load. This is slow, because the docker api was never intended to be used remotely.

Instead kra push is very fast and should be used in CI instead.

github CI example

here's a typical .github/workflows/deploy.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
name: Deploy
on:
  push:
    branches: [  main, 'staging' ]
jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: 'deploy to kraud'
        env:
          KR_ACCESS_TOKEN: ${{secrets.KR_ACCESS_TOKEN}}
        run: |
          curl -L https://github.com/kraudcloud/cli/releases/latest/download/kra-linux-amd64.tar.gz  | tar xvz
          sudo mv kra /usr/local/bin/

          # get credentials for docker cli
          kra setup docker

          # build the images localy
          docker compose build

          # push images to kraud
          kra push

          # destroy running pod
          # this is currently needed until we fix service upgrades
          docker --context kraud.aep rm -f myapp

          # start new pods with new images
          docker compose up -d

kra is open source and available here: https://github.com/kraudcloud/cli. We're aways happy for feeedback. Feel free to open github issues or chat with a kraud engineer on discord.

screenshot

deployment screenshot

global file system is now generally available

Global file system can be mounted simultaneously on multiple containers, enabling easy out of the box shared directories between services.

Similar to NFS, it does make files available over the network, but GFS is fully managed and does not have a single point of failure. It is also significantly faster than NFS due to direct memory access in virtiofs.

1
2
docker volume create shared --driver gfs
docker run -ti -v shared:/data alpine

Coherent filesystem for horizontal scaling

GFS enables an application developer to start multiple instances of the same application, without implementing synchronization. This is specifically useful for traditional stacks like PHP where horizontal scaling requires a separate network storage.

Any docker container works with GFS without changes. The same standard syntax used to mount volumes on your local computers dockers will simply work in the cloud.

Shared object storage for multiple services

Modern applications often choose to store shared files in object storages, specifically s3. With GFS, you can simply store a file using unix file semantics without the need for a separate layer.

File i/o behaves identical using a docker volume on your local machine, and with kraud. This makes developing apps locally and deploying into the kraud seamless.

Built in redundancy by ceph

GFS is backed by cephfs on a 3 times redundant SSD cluster. Ceph is an open source object storage cluster backed by redhat, CERN and others. All pods/containers launched in the Falkenstein DC enjoy a 20MB/s transfer rate.

Users with hybrid regions should note that GFS data transfer counts towards external traffic.

Additionally, customers may choose a separate cluster for large intermediate data on magnetic disks. This is intended for science applications working with large data sets and can easily scale to multiple petabytes.

see the documentation for details