As we know, Linux supports virtual memory. Actually, almost all modern general operating systems such as Solaris, Windows, Mac OS X support virtual memory. Every user space process in Linux has its own virtual address space. The virtual address will be translated to physical address by operating system finally. In fact, most CPU architectures such as x86 and arm provide hardware support for virtual to physical address translation with MMU. In that case, the translation is done by the cooperation of operating system(software) and CPU(hardware). In this post, I am trying to explain the pagemap interface which is used to explore the mapping information of physical memory.
Install SystemTap in Ubuntu 14.04
About 2 months ago, I happened to read a book about how to learn OS principles by hands-on experience on Solaris with MDB and DTrace. After reading the chapters about memory management, I was totally impressed by MDB and DTrace. They acts like a microscope to help you look into the kernel. For example, you can walk through the data structures used by Solaris kernel to manage the process. You can even observe the memory related data structures and emulate the translation between virtual address and physical address manually. Then I became curious about if Linux has similar tools because I am pretty sure that it will be very helpful to understand the design and implementation of Linux kernel.
Use GDB to Understand FUSE File System
During the past years, I have been involved in several storage related projects from which I have learned a lot about the world of storage. Definitely the journey is interesting. However, it is not awesome because almost all the work I have been involved is about applications of storage. Then I decided to dig more deeper in the storage world. It turned out that it is really wonderful to explore in the world of storage.
Actually I have heard about FUSE before but did not spend time to explore it because I thought FUSE would bring a lot of overhead and hence won’t get too much application in system design. During the past months, I have seen the implementations of a few storage solutions, both open source and commercial ones. To my surprise, FUSE is used in those solutions. It is time to get acquainted with it. Per my experience, debugging is the very very useful way to learn a new technology. In this post, I will show how to use GDB to debug the hello example. You can set break points, use single step execution to observe the behavior of the FUSE application.
Avro Cookbook : Part III
- Recipe 6: Serialize data as JSON data
- Recipe 7: Deserialize JSON data
- Recipe 8: Serialize array in JSON
- Recipe 9: Deserialize JSON array data
- Recipe 10: Deserialize data stream
- Summary
Recipe 6: Serialize data as JSON data
In Avro Cookbook : part I, if you open the file /tmp/log created by recipe 3, you would find that it is definitely not a human readable text format. Avro provides the encoder/decoder mechanism which helps to serial the data to text format as JSON data.
Avro Cookbook : Part II
Recipe 5: Serialize data without Code Generation
In formal recipes, before using Avro to serialize/deserialize data, schema files have to be defined to be leveraged by Avro code generation facility to generate the Java classes. This is also recommended when using Avro in Java. However, it is not required. Actually, you can parse the schema on the fly without code generation.
Avro Cookbook : Part I
- Initialize the project structure
- Tweak the pom.xml
- Import the project into Intellij IDEA
- Recipe 2: Define a Schema
- Recipe 3: Serialize the Log Data to Disk File
- Recipe 4: Deserialize the Log Data from Disk File
Avro is a data serialization framework. It is an Apache project led by Doug Cutting who is also the author of several other open source projects such as Hadoop, Lucene. Recently I need to leverage Avro to serialize/deserialize some data, however, I found its document is too poor, at least too poor for newbies like me who don’t have much experience on data exchange format frameworks.
In fact, it is very easy to understand what Avro can do. It helps to convert Java objects into bytes and vice versa. The key information the framework needs to know is the format of the date, namely ‘Schema’ in Avro. In this article, I won’t spend any time on explaining what Avro is.
jetty-maven-plugin and https support
Introduction
jetty-maven-plugin(formerly known as maven-jetty-plugin)is a maven plugin which enables the jetty container in maven project. It is very convenient in development environment since you can start the web app by issue only one command mvn jetty:run. By default it only supports http protocol. Additional configuration is necessary if https is required. Roughly we needs:
- A development certificate
- Instruct the jetty plugin to use the certificate
Besides jetty-maven-plugin, another maven plugin called keytool-maven-plugin is also needed to generate the certificate.
It should be noted that the plugin has evolved a lot since version 7. The configuration is different from different version.
Tips about Octopress
Hello world
The first post written by markdown!
Test if a value implements an interface in golang
We all know that type assert could be used to cast a interface type variable to its actual type value. In the other way, golang allows you to query if a value has implement an interface. This could be used when writing unit test cases, for example, you want to make sure that a struct actually implements certain interfaces.