Similar to Docker CLI, the kubectl is also not very user friendly, you have to be explicit about lots of options which actually make sense as defaults in everyday usage and debugging when we are working on multi-namespaced Kubernetes cluster(s). The aim of this post is to define some functions, aliases, and JSON templates to make the CLI easier for us.
The below screenshot shows a glimpse of what it would enable you to do:
Let’s first look at most frequent kubectl commands and their frequently used options (make sure to pay attention to comments):
The Docker CLI is not very user friendly, you have to be explicit about lots of options which actually make sense as defaults in everyday usage and debugging when we are working in a multi-container environment. The aim of this post is to define some functions, aliases, and JSON templates to make the CLI easier for us.
Docker CLI (client) is a wrapper on top of Docker API which enables us to send Docker API calls down to Docker Daemon.
Let’s first look at most frequent used Docker CLI commands are their frequently used options (make sure to pay attention…
If you are an entrepreneur, you will be successful if you build what solves customers’ problems. You need to be in their shoes, understand their short-term and long-term requirements, need to understand how is their day-to-day life, what problems do they face; and then based on it, prepare short-term and long-term plans to build/enhance the product.
Smiling to your customers or providing them freebies doesn’t help in getting them to stick to your product!
Terraform is IaC (Infrastructure as Code) tool to provision and manage any cloud, infrastructure, or service. It follows a declarative-style and push-type model. It is masterless by default and does not require you to install any extra agents.
Disclaimer: This is going to be a lengthy writeup, as we are going to cover almost everything that you need to know to get started, deploy with best practices, troubleshoot, and some tips on figuring out the right configuration.
Table of Contents
· Terraform References
· Getting Started with Terraform
· Terraform Installation
· Terraform CLI
∘ Terraform Help
∘ The terraform init
∘ The terraform plan
∘…
K8s is awesome, but at the same time, it’s complicated!
K8s is Awesome!
Kubernetes (K8s) is an open-source system for automating deployment, scaling, and management of containerized applications. So it’s a container orchestration tool, but it is just not limited to it as does storage orchestration, service discovery, load balancing, automated rollouts, brings in self-healing, provides secret and configuration management, provides horizontal scaling, and is a declarative way to define a cluster state.
The biggest use-case for K8s is that it encourages micro-service based architecture. It does so by allowing micro-services to be independently scaled.
But at the same time, it’s complicated!
As you scale up K8s, bring in more services, the complexity grows dramatically.
Few prominent issues that I…
Writing and debugging Jsonnet code is not straight-forward, but let’s see what we have at our disposal.
As a data scientist and a software engineer, I have been using YAML files for defining jobs to train machine learning models. Recently, I switched to, data templating lazy language by Google, to DRYup (Don’t Repeat Yourself) the configuration code. The most important benefit that I got was to be able to reuse the templates and hence less maintenance.
Here is how one with knowledge of any programming language (such as Python), can get up to speed with Jsonnet.
Instead of TDD (Test Driven Development) which forces you to think about tests first, one can practice TPD (Test Paralleled Development). In TPD, you start with code development, but during development, you check the correctness of the code by writing tests and executing them (instead of running the code directly or using the console).
Table of Contents
· The unittest module: Unit Testing Framework
∘ _The unittest Basics
∘ _Running Tests using unittest Module
· The unittest.mock module: Mock Object Library
∘ _Mock Class: Mocking objects and/or an attribute
∘ _The MagicMock Class
∘ _Patching imports with patch
∘ _Mock Helpers
The unittest module: Unit Testing Framework
References:
Orchestrate parallel jobs on K8s with the container-native workflow engine.
Table of Contents
∘ Argo CLI
∘ Deploying Applications
∘ Argo Workflow Specs
Argo Workflows is an open-source container-native workflow engine for orchestrating parallel jobs on K8s. Argo Workflows are implemented as a K8s CRD (Custom Resource Definition). As a result, Argo workflow can be managed using kubectl and natively integrates with other K8s services such as volumes, secrets, and RBAC. Each step in the Argo workflow is defined as a container.
- Define workflows where each step in the workflow is a container.
- Model multi-step workflows as a sequence of tasks or capture…
JSON Schema and OpenAPI can seem similar but have different use-cases.
To begin, how JSON Scheme and OPenAPI differ? Well, in contrast to JSON Schema, an OpenAPI document is a definition for an entire API, not just data models. One might compare JSON Schema with the OpenAPI data model.
Why the need to validate JSON?
There are a plethora of use-cases, but let me explain why I use it:
Enter the world of Kubernetes and you’ll find yourself surrounded by object manifests which are either defined as YAML or JSON. But having to maintain thousands of such manifests can be a nightmare if your code is…
Software Engineering, Systems
Don’t get confused with these two similar but different patterns, and know which one to use when.
This difference is important to not just generic Software Engineers, but also to Data Engineers and is the basis for the understanding event-driven architectures for data pipelines.
Let’s look at both of them individually, before we eventually list-out the differences.
Observer Pattern
“The observer pattern is a software design pattern in which an object, called the subject, maintains a list of its dependents, called observers, and notifies them automatically of any state changes, usually by calling one of their methods.” — Wikipedia definition [1]
The…
Munish Goyal
Designing and building large scale applications/APIs, ambitious data models, and workflows! https://goyalmunish.github.io
