Jeff Li

Be another Jeff

Keep Python gRPC Client Connection Truly Alive

Introduction

gRPC is a high performance RPC framework from Google which helps to reduce the gap between components within large scale systems. It provides a program paradigm in which code written by various languages is able to work together as a single language. The framework is promising due to the microservices tide at the moment.

I have been working on a project which leverages gRPC recently. In the project, a component, the gRPC client, written with Python communicates with another one, the gRPC server every day like a daily cron job. And during each round, multiple calls will be issued by the client.

It worked well on the first round. One day later, when 2nd round got scheduled, unavailable exception was observed from the first call with the complaint.

Notes on Ansible Include

Introduction

Include mechanism is one of my favorite features about Ansible because it is the key to adhere to the principle of DRY that stands for Don’t Repeat Yourself which is of great importance to write reusable and maintainable Ansible code.

This post won’t cover every piece of the include feature. Please go through the docs from Ansible docs site before continuing if you are not familiar with Ansible include. link 1, link 2.

A Pitfall of Vagrant When Using Ansible’s wait_for Module

Introduction

Vagrant and Ansible are used to manage my development environment which is really wonderful. Recently, I have written a Ansible playbook which would reboot the VM during Vagrant provisioning. Check here to see how Vagrant works together with Ansible. Besides, there are tasks left which are expected to continue after the VM comes back. It is easy to find a solution in Ansible

Install Protocol Buffer from Source in CentOS 7

Figuring out dependencies in specific distribution is really boring, not fun at all.

Hope this post can save you some time.

1
2
3
4
5
6
sudo yum install autoconf automake libtool unzip gcc-c++ git -y
git clone https://github.com/google/protobuf.git
cd protobuf
./autogen.sh
./configure
make

A Beginner’s Guide for Vagrant

Introduction

Vagrant is a tool developed at HashiCorp intended to relieve the effort to setup development environment. According to its document, Vagrant is able to “create and configure lightweight, reproducible, and portable development environments”. It relizes the vision through virtualization technologies like VirtualBox. By the way, besides Vagrant, HashiCorp also deliver several open source DevOps tools which are really cool. I suggest spending some time on exploring their products if you are interested in DevOps.

Getting Started with RocksDB in CentOS 7

Introduction

RocksDB is a high performance embedded key value storage engine which is written in C++. Though its name tells that it is a database, it is actually a C++ library providing a bunch of API instead of a Client-Server architected database.

I am not a C++ programmer and when trying to play with RocksDB following the Getting Started guide from the RocksDB website, I found it is not easy to run the example in CentOS 7 with either dynamic or static linkage. To be more specific with dynamic linkage, the search paths of dynamic libraries vary from Linux distributions but the installation script of RocksDB does not take care of that.

This post is intended for those who want to play with RocksDB but failed to run the example. What will be mentioned including

  • How to install RocksDB as dynamic library from source in CentOS 7
  • How to install RocksDB as static library in CentOS 7
  • How to run the example program

Build Latest Wireshark in CentOS 7

As we know, Wireshark has switched from GTK+ to Qt since Version 2. As a Mac and Kubuntu user, the GTK UI was really not harmony with my desktop environments. So after the version 2 was released last year, I upgraded Wireshark in both my Mac and Kubuntu boxes.

For some reason, I have to work on some CentOS 7 boxes in which the latest Wireshark in the yum repo is 1.10. Besides, it is difficult to find any existing Wireshark 2 rpm package in the web. After spending some time on the Wireshark document, I came to know that it is really easy to build Wireshark rpm package only if the dependencies are met.

1
2
3
4
5
sudo yum install gcc gcc-c++ bison flex libpcap-devel qt-devel gtk3-devel rpm-build libtool c-ares-devel qt5-qtbase-devel qt5-qtmultimedia-devel qt5-linguist desktop-file-utils
wget https://1.eu.dl.wireshark.org/src/wireshark-2.0.5.tar.bz2
tar xf wireshark-2.0.5.tar.bz2
./configure
make rpm-package

The RPM files could be found in the package/rpm/RPMs/x86_64 directory.

Hope this note could save you some time.

Updates 1 Wireshark 2.2.X has an extra dependency. If configure script complains about

1
configure: error: I couldn't find lrelease-qt5 or lrelease; make sure it's installed and in your path

, it means qt5-linguist which has been mentioned in the comments by Michel, is missing. Above guide has been updated.

Processes Relation on Python Multiprocessing Module

In 2007, when I was taking the undergraduate operating system course, the textbook1 mentioned that a child process could possible be terminated on its parent’s exit. Years later, I came to the world of Linux and found that the design is OS-dependent because in Linux, the child process will keep running even though its parent is gone. But it will be adopted by the process 1 instead of becoming an orphan. Let’s use a small program2 to validate the behaviour.

Dump PTE with SystemTap

In my last blog post, we explored the pagemap interface in Linux which is an interface that exposes some kernel information about the memory management. As I mentioned in that post, the pagemap interface does not expose the raw pte content. Since PTEs are allocated in the kernel space, it is impossible to explore the PTEs from user space applications. To access the kernel space memory, a traditional way is writing a kernel module which, I believe, is of course kind of tedious.

In this article, we’ll see how to leverage SystemTap infrastructure to dump the PTE for the page where an given virtual address of a process lives in.