Zeyuan Hu's Page

Postmortem on TreeTracker Join: Simple, Optimal, Fast

Zeyuan Hu — Sat, 03 Jan 2026 23:01:02 +0800

After five years of blood, sweat, and tears, TreeTracker Join (\(\mathsf{TTJ}\)) has finally been published. The research itself is very interesting and kept me hooked for five years. As a postmortem, I reflect on some of what I learned during the process. I’m hoping this post can be …

Looking for Collaborators

Zeyuan Hu — Sun, 24 Mar 2024 20:11:00 +0800

Motivation
What I’m looking for

Motivation

Quite often, I stumble upon personal websites from academia with sentence like “I’m always in search of collaborators”. I figured I may try to write a sentence like that somewhere on my page as well. I think the most important reason for …

Release of sphinxcontrib-pseudocode

Zeyuan Hu — Tue, 21 Dec 2021 02:11:00 +0800

To celebrate the release of sphinxcontrib-pseudocode, the first sphinx-doc extension I have ever written, I document some implementation details behind this extension.

Introduction
Background
Implementation Details
Development Environment Setup
Conclusion

Introduction

sphinxcontrib-pseudocode allows one to write \(\LaTeX\) algorithm (using algpseudocode …

Build My First PC

Zeyuan Hu — Wed, 29 Sep 2021 02:11:00 +0800

Requirement
Background
Tools
My Choice
Reference

This page contains my note on how to build a PC.

Requirement

I use this PC mostly for research. The requirement is listed below:

Large memory. 32GiB at least …

"Synthesizing Data Structure Transformations from Input-Output Examples"

Zeyuan Hu — Thu, 28 Jan 2021 04:20:00 +0800

The paper presents a method to synthesize functional programs that transform recursive data structures (e.g., lists, trees)
- Examples: see Figure 6 (e.g., join, cprod)
- essentially shows how one can orchestrate a list of operators to generate the desired program
Three techniques
- type-aware inductive generalization
  - purpose: create hypotheses that …

Graph Data Models

Zeyuan Hu — Thu, 14 Jan 2021 23:20:00 +0800

It has been a very long time since my last post. As you might know, I left AWS and started to work on my PhD. Research takes almost all my time. I have written a lot for my research, which consumes all my blogging energy. However, I decided to try …

Status Update

Zeyuan Hu — Wed, 04 Mar 2020 12:20:00 +0800

As you all probably have noticed, I have been quiet since October last year. Here is a status update on my side: September last year, I was laid off due to the strategic change of my last employer and I was busy with job hunting. Luckily, I’m able to …

Semaphore

Zeyuan Hu — Sun, 20 Oct 2019 17:10:00 +0800

Concurrency is a big topic that I’m planning to write about more for the upcoming days. In this post, I’ll cover the concept “semaphore”, a very important concept when we talk about synchronization. I’ll walk through concept and offer an implementation of semaphore in C++ with a …

Secure Connection with MariaDB: A Conceptual Approach

Zeyuan Hu — Sat, 14 Sep 2019 23:20:00 +0800

In this post, I discuss how we can have a secure connection with MariaDB by first understanding the computer science fundamentals behind the steps. Once we have built the concept model, the steps linked above are self-explanatory. In addition, this post applies the same concept to understand SSL ccertificate.

Concepts …

"Ceph: A Scalable, High-Performance Distributed File System"

Zeyuan Hu — Mon, 01 Jul 2019 16:20:00 +0800

Problem
Background
System Design
Additional Reading

Problem

How can we design a distributed file system that:

are scalable (e.g., supports hundreds of petabytes and beyond; extreme workload case)
flexible to adjust to different workloads

while maintaining good performance?

Background

Object-based storage (an abstraction layer between application and hard disks …

"Mnemosyne: Lightweight Persistent Memory"

Zeyuan Hu — Tue, 07 May 2019 12:20:00 +0800

Problem
Background
System Design

Problem

How can we design programming interface for persistent memory (i.e., storage-class memory)?

Background

Storage-class memory (SCM) provides interface of memory (load and store instructions) but the persistence of disks

System Design

Expose SCM as a persistent memory abstraction to provide direct access to the …

"Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing"

Zeyuan Hu — Mon, 06 May 2019 20:24:00 +0800

Problem
System Design
- Goal
- Resilient Distributed Datasets (RDD)

Problem

How to design a system that support in-memory computation (with real-time interaction support) in large cluster efficiently with fault-tolerance?

Prior systems are lack of abstraction for leveraging distributed memory (intermediate computing result reuse is problem: has to save and then read …

"GPUfs: Integrating a File System with GPUs"

Zeyuan Hu — Mon, 06 May 2019 12:20:00 +0800

Problem
Background
System Design

Problem

GPU data access needs explicit management by each individual GPU program. How can we provide an illusion similar to VM to physical pages such that all the data access can be done automatically (i.e., without stating that how data should be fetched from host …

"Dandelion: a Compiler and Runtime for Heterogeneous Systems"

Zeyuan Hu — Mon, 06 May 2019 10:00:00 +0800

Problem
System Design
- Overview
- Architecture
  - Dandelion Compilers
  - Dandelion Runtime

Problem

How to design a system that provides programmability for heterogeneous distributed systems?

Challenges are
- Heterogeneous: different programming models, architecture expertise
- Distributed resources: data movement, scheduling
- Concurrency: synchronization, consistency

System Design

Overview

Goals: make it simple for programmers to write high …

"Operating System Transactions"

Zeyuan Hu — Sat, 27 Apr 2019 12:20:00 +0800

Problem
Background
System Design

Problem

How to implement transaction with ACID guarantee in OS to provide concurrency control?

Background

Problems that can be solved by using transactions:
- Security vulnerbilities in the file system that are caused by time-of-check-to-time-of-use (TOCTTOU) race conditions
- Unsuccessful software installation …

"Scheduler Activations: Effective Kernel Support for the User-Level Management of Parallelism"

Zeyuan Hu — Sat, 13 Apr 2019 12:20:00 +0800

Problem
Background
System Design
Reference

Problem

User-level library
- Management in application’s address space
- High performance and very flexible
- Lack functionality
Operating system kernel
- Poor performance (compared with user-level …

"Jitsu: Just-In-Time Summoning of Unikernels"

Zeyuan Hu — Wed, 13 Mar 2019 21:20:00 +0800

Problem
Background
System Design

Problem

How to build a system that is able to securely manage multi-tenant networked applications on embedded infrastructure?

Goals:
- High density/scalability
- Fast boot
- Lightweight
- VM-level isolation
- “Embedded cloud”

Background

OS are traditionally designed to run on a wide range of hardware, and support a variety …

"Rethinking the Library OS from the Top Down"

Zeyuan Hu — Wed, 13 Mar 2019 20:20:00 +0800

Problem
Background
System Design

Problem

How can we refactor a commerical OS to follow libOS architecture and achieve a better performance than VMM approach?

Background

Three categories of services in OS implementations:

Why does a guest OS need a kernel? Because the host interface is virtual hardware

System Design

Goals …

"Arrakis: The Operating System is the Control Plane"

Zeyuan Hu — Wed, 13 Mar 2019 17:20:00 +0800

Problem
Background
System Design
Remarks

Problem

How can we design an OS for I/O intensive applications such that most I/O operations do not need kernel mediation?

Background

The authors make a classic efficiency argument: servers usually perform conceptually simple operations, but in practice this results in too much …

"Memory Resource Management in VMware ESX Server"

Zeyuan Hu — Mon, 11 Mar 2019 23:24:00 +0800

Problem
Background
System Design
Remarks

Problem

How to design a memory management system inside VMM to manage memory allocated to each guest OS (i.e., VM)? This is challenging as each guest OS also has its own resource manager and how …

"Xen and the Art of Virtualization"

Zeyuan Hu — Mon, 11 Mar 2019 23:13:00 +0800

Problem
Background
System Design
Further reading

Problem

Provide a high performance resource-managed virtual machine monitor (VMM) that provides performance guarantees to concurrent execution of multiple operating systems: “hosting up to 100 virtual machine in- stances simultaneously on a modern server”

Background

Big picture

Two types of VMMs

Virtualization techniques ^{1 …}

"Practical, transparent operating system support for superpages"

Zeyuan Hu — Sun, 10 Mar 2019 15:00:00 +0800

Problem
Superpage definition and benefits
Superpage issues
System Designs
Remarks

Problem

How to design a general and transparent superpage management system to achieve high and sustained performance for real workloads and negligible degradation in pathological situations?

Superpage definition and …

"Threads and Input/Output in the Synthesis Kernel"

Zeyuan Hu — Sun, 10 Mar 2019 12:20:00 +0800

Problem
System Designs
Remarks

Problem

Design an OS for a parallel and distributed computational environment and achieve the following three goals:

High performance
Self-tuning capability to dynamic load and configuration changes
A simple, uniform and …

"END-TO-END ARGUMENTS IN SYSTEM DESIGN"

Zeyuan Hu — Sat, 09 Mar 2019 21:20:00 +0800

Main Point

The following statements provide different angles of stating the same “end-to-end argument”:

Functions placed at low levels of a system may be redundant or of little value when compared with the cost of providing them at that low level; low level mechanisms to support these functions are justified …

"Exokernel: An Operating System Architecture for Application-Level Resource Management"

Zeyuan Hu — Sat, 09 Mar 2019 13:20:00 +0800

Problem

How can we design an OS …

"The UNIX Time- Sharing System"

Zeyuan Hu — Sun, 03 Mar 2019 11:13:00 +0800

Problem
System designs
Remarks

Problem

How to design a interative-use system with easy use (i.e., write, test, and run programs) given the hardware constraint?

System designs

Architecture

File System

Untyped data (byte oriented)
- Structure of files is controlled by the programs which uses them …

Brush up OS

Zeyuan Hu — Wed, 23 Jan 2019 22:24:00 +0800

This post aims to prepare myself for the upcoming CS380L Advanced Operating Systems offered by Christopher J. Rossbach. The questions are actually from his HW1 aka. “Swapping in the state from undergraduate OS”.

Brush up

Modify char in another function

Zeyuan Hu — Wed, 26 Sep 2018 10:00:00 +0800

Almost two years ago, I write a post on how to modify an array in one function through another function in C. I did pretty detailed study through GDB there but I find that the illustration is lengthy to read. In this post, I try to show the same concept …

Graph basics + Topological Sort

Zeyuan Hu — Sat, 14 Jul 2018 22:30:00 +0800

Basic graph concepts

Flash-based SSD Basics

Zeyuan Hu — Sun, 01 Jul 2018 22:00:00 +0800

list out the basic concepts in SSD

Generalized binary search

Zeyuan Hu — Sun, 10 Jun 2018 01:24:00 +0800

Binary search idea can be generalized to other problems

Trie

Zeyuan Hu — Sat, 02 Jun 2018 22:30:00 +0800

A study of trie data structure

"Weighted Voting for Replicated Data"

Zeyuan Hu — Tue, 15 May 2018 12:20:00 +0800

“Weighted Voting for Replicated Data” paper reading

"The Andrew File System (AFS)"

Zeyuan Hu — Wed, 02 May 2018 23:30:00 +0800

“The Andrew File System (AFS)” paper reading

"Petal: Distributed Virtual Disks"

Zeyuan Hu — Wed, 02 May 2018 02:20:00 +0800

“Petal: Distributed Virtual Disks” paper reading

"Sun's Network File System (NFS)"

Zeyuan Hu — Tue, 01 May 2018 02:20:00 +0800

“Sun’s Network File System (NFS)” paper reading

"MapReduce: Simplified Data Processing on Large Clusters"

Zeyuan Hu — Mon, 30 Apr 2018 23:20:00 +0800

“MapReduce: Simplified Data Processing on Large Clusters” paper reading

"Scaling Distributed Machine Learning with the Parameter Server"

Zeyuan Hu — Mon, 30 Apr 2018 01:30:00 +0800

“Scaling Distributed Machine Learning with the Parameter Server” paper reading

"Dynamo: Amazon’s Highly Available Key-value Store"

Zeyuan Hu — Sat, 14 Apr 2018 20:24:00 +0800

“Dynamo: Amazon’s Highly Available Key-value Store” paper reading

"Fast Crash Recovery in RAMCloud"

Zeyuan Hu — Mon, 09 Apr 2018 22:00:00 +0800

“Fast Crash Recovery in RAMCloud” paper reading

"PebblesDB: Building Key-Value Stores using Fragmented Log-Structured Merge Trees"

Zeyuan Hu — Fri, 30 Mar 2018 00:45:00 +0800

“PebblesDB: Building Key-Value Stores using Fragmented Log-Structured Merge Trees” paper reading

"BLEU: a Method for Automatic Evaluation of Machine Translation"

Zeyuan Hu — Wed, 28 Mar 2018 01:24:00 +0800

“BLEU: a Method for Automatic Evaluation of Machine Translation” paper reading

"Existential Consistency: Measuring and Understanding Consistency at Facebook"

Zeyuan Hu — Thu, 08 Mar 2018 20:24:00 +0800

“Existential Consistency: Measuring and Understanding Consistency at Facebook” paper reading

"PNUTS: Yahoo!’s Hosted Data Serving Platform"

Zeyuan Hu — Thu, 08 Mar 2018 20:24:00 +0800

“PNUTS: Yahoo!’s Hosted Data Serving Platform” paper reading

Cache, Lease, Consistency, Invalidation

Zeyuan Hu — Wed, 07 Mar 2018 20:24:00 +0800

Cache, Lease, Consistency, Invalidation

State Machine Replication Approach

Zeyuan Hu — Wed, 07 Mar 2018 20:24:00 +0800

State Machine Replication Approach

Lamport Clocks, Vector Clocks

Zeyuan Hu — Tue, 06 Mar 2018 20:24:00 +0800

Lamport Clocks, Vector Clocks

Distributed System Reference Guide

Zeyuan Hu — Tue, 06 Mar 2018 16:24:00 +0800

"Why do computers stop and what can be done about it?"

Zeyuan Hu — Sun, 21 Jan 2018 20:24:00 +0800

“Why do computers stop and what can be done about it?” paper reading

"Introduction to Distributed System Design"

Zeyuan Hu — Wed, 17 Jan 2018 16:24:00 +0800

“Introduction to Distributed System Design” paper reading

How to write binary search correctly

Zeyuan Hu — Fri, 12 Jan 2018 16:24:00 +0800

How to write a binary search correctly on the first try

Crowdsourcing readings

Zeyuan Hu — Sat, 30 Dec 2017 12:30:00 +0800

CS 395T Human Computation / Crowdsourcing papers reading

Introduction to Conditional Random Fields

Zeyuan Hu — Fri, 22 Sep 2017 10:20:00 +0800

Note

This is a repost of the Edwin Chen‘s blog: Introduction to Conditional Random Fields in year 2012. I do this based on three purposes: 1. Bookmark for my course project reference 2. Fix the math rendering issue happened to the original post 3. Small tweaks to the layout …

Watching log of CMU Database Systems course

Zeyuan Hu — Fri, 08 Sep 2017 12:30:00 +0800

log book for CMU 15-445/15-645 Database Systems Fall 2017 course

MAW Chapter 8: Disjoint set

Zeyuan Hu — Tue, 29 Aug 2017 01:12:00 +0800

disjoint set Union-Find algorithm

Knapsack problem

Zeyuan Hu — Mon, 14 Aug 2017 16:45:00 +0800

summarizes knapsack problem and its variations

Understanding how function call works

Zeyuan Hu — Sun, 30 Jul 2017 00:21:00 +0800

Understand call stack in programming language (i.e. C)

Andrew Ng's ML Week 06, 11

Zeyuan Hu — Fri, 21 Jul 2017 12:51:00 +0800

Advice on building a machine learning system from course

Virtual methods and polymorphism in C++

Zeyuan Hu — Wed, 12 Jul 2017 23:11:00 +0800

virtual, pure virtual, constructors, abstract base class

Merge sort

Zeyuan Hu — Sat, 01 Jul 2017 23:33:00 +0800

summary for mergesort

join in SQL

Zeyuan Hu — Fri, 23 Jun 2017 21:23:00 +0800

A study on join statement in SQL

The tortoise and the hare

Zeyuan Hu — Sun, 18 Jun 2017 20:20:00 +0800

Floyd’s Tortoise and Hare w/ leetcode 142, 287

Python case study: leetcode scraper

Zeyuan Hu — Thu, 15 Jun 2017 21:22:00 +0800

a record of python usage appears in the leetcode scraper

Solving recurrence relations (part 2)

Zeyuan Hu — Mon, 12 Jun 2017 17:20:00 +0800

Additional ways to solve recurrence relation

Draw a Neural Network through Graphviz

Zeyuan Hu — Thu, 25 May 2017 22:20:00 +0800

Use NN as an example to show graphviz tricks

Andrew Ng's ML Week 04 - 05

Zeyuan Hu — Tue, 23 May 2017 22:20:00 +0800

Neural network

Andrew Ng's ML Week 01 - 03

Zeyuan Hu — Fri, 05 May 2017 16:18:00 +0800

ML overview, Linear regression, Logistic regression, Regularization

Shell sort

Zeyuan Hu — Mon, 01 May 2017 21:33:00 +0800

summary for shell sort

Simple sorting algorithms

Zeyuan Hu — Mon, 24 Apr 2017 21:33:00 +0800

bubble sort, selection sort, and insertion sort

Introducing the "Andrew Ng's ML course study notes"

Zeyuan Hu — Fri, 21 Apr 2017 23:48:00 +0800

Let the battle begins :)

Sorting prelim

Zeyuan Hu — Tue, 18 Apr 2017 10:03:00 +0800

Prelim for sorting

MAW Chapter 7: Sorting writing questions

Zeyuan Hu — Sat, 15 Apr 2017 23:54:00 +0800

My solutions to selected problems in MAW Chapter 7

MAW: Chapter 6 Reflection

Zeyuan Hu — Sun, 09 Apr 2017 10:45:00 +0800

An overview summary of MAW Chapter 6

Binomial queue

Zeyuan Hu — Sat, 08 Apr 2017 23:33:00 +0800

Summary for binomial queue

Skew heap

Zeyuan Hu — Wed, 05 Apr 2017 23:33:00 +0800

Summary for skew heap

Leftist heap

Zeyuan Hu — Tue, 04 Apr 2017 10:30:00 +0800

Summary for leftist heap

Binary heap

Zeyuan Hu — Sun, 02 Apr 2017 11:30:00 +0800

Binary heap summary

MAW Chapter 6: Priority Queues (Heaps) writing questions

Zeyuan Hu — Sun, 26 Mar 2017 12:01:00 +0800

My solutions to selected problems in MAW Chapter 6

Hash Table

Zeyuan Hu — Fri, 17 Mar 2017 15:56:00 +0800

Summary of hash table for MAW Chapter 5

MAW Chapter 5: Hashing writing questions

Zeyuan Hu — Thu, 16 Mar 2017 17:41:00 +0800

My solutions to selected problems in MAW Chapter 5

MAW: Chapter 4 Reflection

Zeyuan Hu — Sun, 12 Mar 2017 10:45:00 +0800

An overview summary of MAW Chapter 4

B-Tree

Zeyuan Hu — Sat, 11 Mar 2017 21:32:00 +0800

B-tree summary

Splay Tree

Zeyuan Hu — Mon, 13 Feb 2017 01:12:00 +0800

Splay tree summary

AVL Tree

Zeyuan Hu — Sun, 05 Feb 2017 10:43:00 +0800

AVL tree summary

Solving recurrence relations in a nutshell

Zeyuan Hu — Thu, 02 Feb 2017 01:05:00 +0800

MATH475 methods to solve recurrence relation

Binary Tree & Binary Search Tree

Zeyuan Hu — Sun, 29 Jan 2017 13:20:00 +0800

Binary tree and binary search tree summary

MAW Chapter 4: Tree writing questions

Zeyuan Hu — Thu, 26 Jan 2017 17:41:00 +0800

My solutions to selected problems in MAW Chapter 4

Tree Terminology

Zeyuan Hu — Tue, 24 Jan 2017 20:23:00 +0800

Commonly seen tree terminology

MAW: Chapter 3 Reflection

Zeyuan Hu — Mon, 23 Jan 2017 10:45:00 +0800

An overview summary of MAW Chapter 3

Num of function calls in recursive Fibonacci routine

Zeyuan Hu — Sun, 22 Jan 2017 23:12:00 +0800

Origin from MAW 3.24

Typecasting in C

Zeyuan Hu — Thu, 19 Jan 2017 00:53:00 +0800

A study of typecasting in C

Difference between i++ and ++i

Zeyuan Hu — Wed, 11 Jan 2017 22:39:00 +0800

Understanding easily confused part of C

Modify array inside function in C

Zeyuan Hu — Sun, 08 Jan 2017 09:23:00 +0800

Commonly seen C usage

Josephus Problem & Radix Sort Reflection

Zeyuan Hu — Fri, 06 Jan 2017 00:34:00 +0800

Takeaway from MAW 3.13 and 3.10

Josephus problem

Zeyuan Hu — Sat, 31 Dec 2016 20:24:00 +0800

Preface

This is actually MAW 3.10. I gradually realize how dense MAW is. In the previous problem, I write almost 500 lines of code. For this one, the problem is not really diffcult to solve if we implement a program that follows the game rule exactly. However, I figure …

A peek in code optimization

Zeyuan Hu — Wed, 28 Dec 2016 13:21:00 +0800

MAW 3.9 solution optimization

Reflection on integer arithmetic package problem

Zeyuan Hu — Mon, 26 Dec 2016 23:03:00 +0800

Origin from MAW 3.9

A small C trick I learned today

Zeyuan Hu — Sat, 24 Dec 2016 23:11:00 +0800

A C one-liner

Print singly linked list in reverse order

Zeyuan Hu — Fri, 23 Dec 2016 00:05:00 +0800

See above

Environment variable substitution using Sed

Zeyuan Hu — Wed, 21 Dec 2016 12:07:00 +0800

A sed example

What's the difference between sourcing a script and executing a script?

Zeyuan Hu — Tue, 20 Dec 2016 21:49:00 +0800

Amusing question in shell

Polynomial Multiplication

Zeyuan Hu — Sun, 18 Dec 2016 18:53:00 +0800

Origin from MAW 3.7

Pelican Hack Day

Zeyuan Hu — Sat, 17 Dec 2016 22:44:00 +0800

Design log for Pelican website

Lesson Learned: Portability

Zeyuan Hu — Fri, 16 Dec 2016 23:20:00 +0800

Shell is not portable!

Reverse Singly Linked List

Zeyuan Hu — Sat, 03 Dec 2016 20:34:00 +0800

MAW 3.12 question

PrintLots

Zeyuan Hu — Mon, 28 Nov 2016 18:20:00 +0800

MAW 3.2

Automatically publish Tinkerer bld output to GitHub with Travis CI

Zeyuan Hu — Sun, 27 Nov 2016 22:00:00 +0800

A taste of DevOps

Generate a Linked List from a given array

Zeyuan Hu — Sun, 27 Nov 2016 19:38:00 +0800

Where MAW starts

Hello World

Zeyuan Hu — Wed, 23 Nov 2016 23:00:00 +0800

Hello

Minimal Emacs Tutorial

Zeyuan Hu — Sun, 18 Oct 2015 16:18:00 +0800

Emacs quick reference