Kubernetes is Evolving, Are You?

Wide-spread adoption of Kubernetes over the last several years has been remarkable and Kubernetes is now recognized as the most popular orchestration tool for containerized workloads. As applications and workflows in Kubernetes continue to evolve, so must the platform and storage.  

So, where are we today, and where are we going? Find out on October 11, 2022 in this webcast “15 Minutes in the Cloud: Kubernetes is Evolving, Are You?,” where we’ll discuss:

Read More

Kubernetes is Everywhere Q&A

Last month, the SNIA Cloud Storage Technologies Initiative hosted a fascinating panel discussion “Kubernetes is Everywhere: What About Cloud Native Storage?”  where storage experts from SNIA and Kubernetes experts from the Cloud Native Computing Foundation (CNCF) discussed storage implications for Kubernetes. It was a lively and enlightening discussion on key considerations for container storage. In this Q&A blog, our panelists Nick Connolly, Michael St-Jean, Pete Brey and I elaborate on some of the most intriguing questions during the session.

Q. What are the additional/different challenges for Kubernetes storage at the edge – in contrast to the data center?  

A. Edge means different things depending on context. It could mean enterprise or provider edge locations, which are typically characterized by smaller, compact deployments of Kubernetes. It could mean Kubernetes deployed on a single node at a site with little or no IT support, or even disconnected from the internet, on ships, oil rigs, or even in space for example. It can also mean device edge, like MicroShift running on a small form factor computer or within an ARM or FPGA card for example.

One big challenge for Kubernetes at the edge in general is to provide a lightweight deployment. Added components, like container-native storage, are required for many edge applications, but they take up resources. Therefore, the biggest challenge is to deploy the storage resources that are necessary for the workload, but at the same time, making sure your footprint is appropriate for the deployment infrastructure.  

Read More

Kubernetes Business Resiliency FAQ

The SNIA Cloud Storage Technologies Initiative continued our webcast series on Kubernetes last month with an interesting look at the business resiliency of Kubernetes. If you missed “A Multi-tenant, Multi-cluster Kubernetes Datapocalypse in Coming” it’s available along with the slide deck in the SNIA Educational Library here. In this Q&A blog, our Kubernetes expert, Paul Burt, answers some frequently asked questions on this topic.

Q: Multi-cloud: Departments might have their own containers; would they have their own cloud (i.e. Hybrid Cloud)?  Is that how multi-cloud might start in a company?

A: Multi-cloud or hybrid cloud is absolutely a result of different departments scaling containers in a deployment. Multi-cloud means multiple clusters, but those can be of various configurations. Different clusters and clouds need to be tuned for the needs of the organization.

Read More

Business Resiliency in a Kubernetes World

At the 2018 KubeCon keynote, Monzo Bank explained the potential risk of running a single massive Kubernetes cluster. A minor conflict between etcd and Java led to an outage during one of their busiest business days, prompting questions, like “If a cluster goes down can our business keep functioning?”  Understanding the business continuity implications of multiple Kubernetes clusters is an important topic and key area of debate.

It’s an opportunity for the SNIA Cloud Storage Technologies Initiative (CSTI) to host “A Multi-tenant Multi-cluster Kubernetes “Datapocalypse” is Coming” – a live webcast on June 23, 2020 where Kubernetes expert, Paul Burt, will dive into:

Read More

Understanding Kubernetes in the Cloud

Ever wonder why and where you would want to use Kubernetes? You’re not alone, that’s why the SNIA Cloud Storage Technologies Initiative is hosting a live webcast on May 2, 2019 “Kubernetes in the Cloud.”

Kubernetes (k8s) is an open-source system for automating the deployment, scaling, and management of containerized applications. Kubernetes promises simplified management of cloud workloads at scale, whether on-premises, hybrid, or in a public cloud infrastructure, allowing effortless movement of workloads from cloud to cloud. By some reckonings, it is being deployed at a rate several times faster than virtualization.

Read More

Wondering What’s New in Container Storage?

The landscape of containers is moving fast and constantly changing, with new standards emerging every few months. If you wondering what’s new in container storage, you are not alone. That’s why the SNIA Cloud Storage Technologies Initiative is hosting a live webcast on February 26, 2019, “What’s New in Container Storage.”

In this webcast, Keith Hudgins of Docker joins us as a follow up to his earlier container webcast “Intro to Containers, Container Storage and Docker.” It’s our most popular webcast to date with thousands of views. If you missed it, it’s available on demand and will provide you with some great background information before our February 26h webcast.

I encourage you to register today for the February 26th session where you’ll learn:

  • What’s new, what to pay attention to, and how to make sense of the ever-shifting container landscape.
  • Container storage types and Container Frameworks
  • An overview of the various storage APIs for the container landscape
  • How to identify the most important projects to follow in the container world
  • The Container Storage Interface spec and Kubernetes 1.13
  • How to get involved in the container community

It will be live, so bring your questions!

A Q&A on Containers and Persistent Memory

The SNIA Cloud Storage Initiative recently hosted a live webcast “Containers and Persistent Memory.” Where my colleagues and I discussed persistent storage for containers, persistent memory for containers, infrastructure software changes for persistent memory-based containers, and what SNIA is doing to advance persistent memory. If you missed the live event, it’s now available on-demand. You can also download a PDF of the webcast slides.

As promised, we are providing answers to the questions we received during the live event.

Q. How is” Enterprise Server SAN” different from “Traditional” Server SAN?

A. Traditional Server SAN refers to individual servers connected to a dedicated, separate SAN storage solution (e.g. EMC VNX, NetApp FAS, etc.); whereas, Enterprise Server SAN refers to the use of direct-attached-storage that is then aggregated across multiple connected servers to create a “virtual SAN” that is not a separate storage solution, but rather benefits from utilizing the existing capacity contained within the application servers, but in a virtualized, shared pool to improve overall efficiency.

Q. Are there any performance studies done with Containers using Tier 1 apps/Business critical?

A. There have been performance characterizations done on Tier 1, Business Critical applications such as Oracle, MySQL and others. However, this would be vendor specific and the user would have to contact and work with each storage vendor to better understand their specific performance capabilities.

Q. Even though Linux and Microsoft support NVDIMM natively, does the MB/BIOS still need to have support?

A. Yes, the MB needs to have the BIOS enabled to recognize NVDIMMs and it needs the ADR signal wired from the Intel CPU to the DIMMs sockets. The motherboard needs to follow the JEDEC standard for NVDIMMs.

Q. If someone unplugs NVDIMM-N and moves it to another server… what will happen?

A. If the system crashed due to a power loss the data in the NVDIMM will be saved. When it is plugged into another NVDIMM-enabled server the BIOS will check if there is saved data in the NVDIMM and restore that data to DRAM before the system continues to boot.

Q. Are traditional storage products able to support containerized applications?

A. Yes, assuming that they support container orchestration engines such as Docker Swarm or Kubernetes through a “container volume plugin.” However, to the extent that they support containerized applications, it is very specific vendor-to-vendor and there are also a number of new storage products that have been developed exclusively to support containerized applications (e.g. Veritas, Portworx, Robin Systems).

Q. How do the storage requirements for containers compare or differ from those of virtual machines?

A. Actually, “production storage requirements” are very similar—albeit almost equivalent—between containerized applications and applications running within virtual machines; the main difference being that due to the scalability potential of containers, these requirements are often exacerbated. Some of these requirements common to both include: data persistence, data recovery, data performance and data security.

Unlock the Power of Persistent Memory in Containers

Containers and persistent memory are both very hot topics these days. Containers are making it easier for developers to know that their software will run, no matter where it is deployed and no matter what the underlying OS is as both Linux and Windows are now fully supported. Persistent memory, a revolutionary data storage technology used in 3d printing london, will boost the performance of next-generation packaging of applications and libraries into containers. On July 27th, SNIA is hosting a live webcast “Containers and Persistent Memory.” Read More

Containers, Docker and Storage – An Expert Q&A

Containers continue to be a hot topic today as is evidenced by the more than 2,000 people who have already viewed our SNIA Cloud webcasts, “Intro to Containers, Container Storage and Docker“ and “Containers: Best Practices and Data Management Services.” In this blog, our experts, Keith Hudgins of Docker and Andrew Sullivan of NetApp, address questions from our most recent live event.

Q. What is the major challenge for storage in containerized environment?

A. Containers move fast. Users can spin up and spin down containers extremely quickly. The biggest challenge in production-bound container environments is simply keeping up with the movement of data.

Docker Engine does not delete base container images when the container is shut down. Likewise, Registry assumes you’ve got unlimited storage on hand. For containers that push frequent revisions (as would be the case in a continuous delivery environment), that leads to a lot of orphaned container images that can fill up all available storage if left unchecked.

There are some community-led scripts that will help to keep things in control. That’s the beauty of community-led technology.

Q. What about the speed of retrieving the data from storage?

A. That’s where being a solid storage architect comes in. Every storage system has different strengths and weaknesses, so it’s important to engineer your solution to fit your performance goals. Docker containers are running on the main kernel of the host system. IO is not constrained by abstraction, as in the case of virtual machines. Rather, it is constrained more by density – hundreds of containers on a host can push massive IOPS, so you want your pipes fat and data sources close to the host systems.

Q. Can you expand on moving Docker Volumes from On-Premise bare metal to Cloud Service Providers? Data Migration? Encryption? 

A. None of these capabilities are built-in to Docker Engine. We rely on external storage systems to provide those features. Private-to-cloud replication is primarily a feature of software-based companies, like Portworx, Blockbridge, or Hedvig. Encryption and migration are both common features across other companies as well. Flocker from ClusterHQ is a service broker system that provides many bolt-on features for storage systems they support. You can also use community-supplied services like Ceph to get you there.

Q. Are you familiar with “Flocker” that apparently is able to copy persistent data to another container? Can share your thoughts?

A. Yes. ClusterHQ (makers of Flocker) provide an API broker that sits between storage engines and Docker (and other dynamic infrastructure providers, like OpenStack), and they also provide some bolt-on features like replication and encryption.

Q. Is there any sort of feature in the volume plugins that allows a persistent volume to re-connect to a container if the container is moved across multiple hosts?

A. There’s no feature in plugins to cover that specifically. The plugin API is very simple. In practice, what you would do is write your plugin to expose volumes to Docker Engine on every host that it’s possible to mount that volume. In your container specification, whether it’s a Compose file, DAB file, or what have you, specify the name of your volume. Wherever that unique name is encountered, it will be mounted and attached to the container when it’s re-launched.

If you have more questions on containers, Docker and storage, check out our first Q&A blog: Containers: No Shortage of Interest or Questions.

I also encourage you to join our Containers opt-in email list. It will be a good way to keep up with all the SNIA Cloud is doing on this important technology.

The Next Step for Containers: Best Practices and Data Management Services

In our first SNIA Cloud webcast on containers, we provided a solid foundation on what containers are, container storage challenges and Docker. If you missed the live event, it’s now available on-demand. I encourage you to check it out, as well as our webcast Q&A blog.

So now that we have set the stage and you’ve become acquainted with basic container technologies and the associated storage challenges in supporting applications running within containers in production, we will be back on December 7th. This time we will take a deeper dive into what differentiates this technology from what you are used to with virtual machines. Containers can both complement virtual machines and also replace them, as they promise the ability to scale exponentially higher. They can easily be ported from one physical server to another or to one platform—such as on-premise—to another—such as public cloud providers like Amazon AWS.

At our December 7th webcast, “Containers: Best Practices and Data Management Services,” we’ll explore container best practices to address the various challenges around networking, security and logging. We’ll also look at what types of applications more easily lend themselves to a microservice architecture versus which applications may require additional investments to refactor/re-architect to take advantage of microservices.

On December 7th, we’ll be on hand to answer your questions on the spot. I encourage you to register today. We hope you can attend!