Background
In order to be able to compose e-mail within meli and avoid writing an editor, making an embed terminal was obviously my only choice. I didn’t bother to look if there are already libraries in Rust for this because as always my prime motivator is figuring things out, for fun.
If you have ever executed tty before you’d probably see a response like /dev/pts/16. pts stands for pseudoterminal slave: the process (such as a shell) running inside it thinks it’s speaking directly to an actual terminal, i.e. the virtual console you see when you Ctrl Alt F1 etc. Terminal emulators such as xterm create pseudoterminals and communicate with them by reading and writing to the pts device and the pseudoterminal reads and writes to /dev/ptmx. The kernel handles the pairing of I/O between each master/slave. You’ll read how to create one below, but for more information read pty(7) on Linux, pty(4) on *BSDs.
A pseudoterminal (sometimes abbreviated “pty”) is a pair of virtual character devices that provide a bidirectional communication channel. One end of the channel is called the master; the other end is called the slave. The slave end of the pseudoterminal provides an interface that behaves exactly like a classical terminal. A process that expects to be connected to a terminal, can open the slave end of a pseudoterminal and then be driven by a program that has opened the master end. Anything that is written on the master end is provided to the process on the slave end as though it was input typed on a terminal. For example, writing the interrupt character (usually control-C) to the master device would cause an interrupt signal (SIGINT) to be generated for the foreground process group that is connected to the slave. Conversely, anything that is written to the slave end of the pseudoterminal can be read by the process that is connected to the master end. Pseudoterminals are used by applications such as network login services (ssh(1), rlogin(1), telnet(1)), terminal emulators such as xterm(1), script(1), screen(1), tmux(1), unbuffer(1), and expect(1).
First steps
Let’s begin with creating the pty:
use crate::terminal::position::Area;
use nix::fcntl::{open, OFlag};
use nix::{ioctl_none_bad, ioctl_write_ptr_bad};
use nix::pty::{grantpt, posix_openpt, ptsname, unlockpt, Winsize};
use nix::sys::stat::Mode;
use libc::TIOCSCTTY;
// ioctl request code to set window size of pty:
use libc::TIOCSWINSZ;
use std::path::Path;
use std::process::{Command, Stdio};
use std::os::unix::io::{FromRawFd, IntoRawFd};
// nix macro that generates an ioctl call to set window size of pty:
ioctl_write_ptr_bad!(set_window_size, TIOCSWINSZ, Winsize);
// request to "Make the given terminal the controlling terminal of the calling process"
ioctl_none_bad!(set_controlling_terminal, TIOCSCTTY);
pub fn create_pty() -> nix::Result<()> {
/* Create a new master */
let master_fd = posix_openpt(OFlag::O_RDWR)?;
/* For some reason, you have to give permission to the master to have a
* pty. What is it good for otherwise? */
grantpt(&master_fd)?;
unlockpt(&master_fd)?;
/* Get the path of the slave */
let slave_name = unsafe { ptsname(&master_fd) }?;
/* Try to open the slave */
let _slave_fd = open(Path::new(&slave_name), OFlag::O_RDWR, Mode::empty())?;
/* Launch our child process. The main application loop can inspect and then
pass the stdin data to it. */
let _child_pid = match fork() {
Ok(ForkResult::Child) => {
/* Open slave end for pseudoterminal */
let slave_fd = open(Path::new(&slave_name), OFlag::O_RDWR, stat::Mode::empty())?;
// assign stdin, stdout, stderr to the tty
dup2(slave_fd, STDIN_FILENO).unwrap();
dup2(slave_fd, STDOUT_FILENO).unwrap();
dup2(slave_fd, STDERR_FILENO).unwrap();
/* Become session leader */
nix::unistd::setsid().unwrap();
unsafe { set_controlling_terminal(slave_fd) }.unwrap();
nix::unistd::execv(
&CString::new("/usr/bin/vim").unwrap(),
Vec::new(),
).unwrap();
/* This path shouldn't be executed. */
std::process::exit(-1);
}
Ok(ForkResult::Parent { child }) => child,
Err(e) => panic!(e),
};
/* Continue dealing with the pty in a thread so that the main application doesn't lock up */
std::thread::Builder::new()
.spawn(move || {
let winsize = Winsize {
ws_row: 25,
ws_col: 80,
ws_xpixel: 0,
ws_ypixel: 0,
};
let master_fd = master_fd.into_raw_fd();
/* Tell the master the size of the terminal */
unsafe { set_window_size(master_fd, &winsize).unwrap() };
let master_file = unsafe { std::fs::File::from_raw_fd(master_fd) };
/* start the pty liaison */
liaison(master_file);
})
.unwrap();
Ok(())
}So far graphics are not in the picture (heh). To draw directly on the screen we can print the bytes we read from the master without doing any sort of work in-between:
fn liaison(pty_fd: std::fs::File) {
let stdout = std::io::stdout();
let mut lock = stdout.lock();
let mut bytes_iter = pty_fd.bytes();
while let Some(Ok(byte)) = bytes_iter.next() {
lock.write_all(&[byte]).unwrap();
}
}Embedding graphics
A quick explanation on how meli graphics work: There’s a two dimensional grid representing the state of the terminal at the current moment. Its elements are of type Cell:
To cache stuff I’ve made some UI components draw their stuff into their own grids and whenever I have to redraw, they copy the cache instead of recomputing data. To actually show changes on the screen each component pushes the area of the grid they have changed into the app state, and whenever it’s time to draw the app displays only the dirty areas. On resize all of the components are marked dirty and redraw everything on the newly resized grid.
The embedded process can thus draw its output on such a grid and we can then draw it like a regular UI widget.
How the terminal handles output
For terminal UI apps we’re interested in the Alternative screen buffer mode. In this mode the application is responsible for handling the cursor. There is no scrollback, to emulate scrolling the app has to redraw the screen entirely or use a special scrolling functionality that is out of scope for this post.
To handle colours, cursor style changes, cursor position changes, the application sends control sequences to the terminal, which are sequences of bytes with designated prefixes and suffixes that correspond to a command. xterm control sequences are not standardised, but here is one list. A quick summary with examples can be found in the Wikipedia Article.
All sequences start with the Escape character, 0x1b, and are sorted in separate categories. The basic one is CSI or Command Sequence Introducer, which is Escape followed by [.
Since we’re pretending to be a terminal, we will encode the code sequence input’s state in a state machine:
pub enum State {
Normal, // Waiting for any kind of byte
ExpectingControlChar, // We just got an ESC and expect a control sequence
Csi, // ESC [ Control Sequence Introducer
/* Multiparameter sequences are of the form KIND P_1 ; P_2 ; ... TERMINATION */
Csi1(Vec<u8>), // CSI with one buffer (CSI <num>) waiting for new buffer digits, a second buffer or a termination character
Csi2(Vec<u8>, Vec<u8>), // CSI with two buffers (CSI <num> ; <num>) as above
Csi3(Vec<u8>, Vec<u8>, Vec<u8>), // CSI with three buffers
CsiQ(Vec<u8>), // CSI followed by '?'
Osc1(Vec<u8>), // ESC ] Operating System Command
Osc2(Vec<u8>, Vec<u8>),
}The embed grid will include all the simulated terminal’s state:
pub struct EmbedGrid {
/*
1 2 ..
┏━━━┯━━━━━━━┅┅ x axis
1 ┃1,1│
┠───┼┈┈
2 ┃ ┊
┃
.. ┃
y axis ┇
*/
cursor: (usize, usize), // xterm cursor starts from (1, 1), but ours from (0, 0)
pub grid: CellBuffer,
pub state: State,
pub stdin: std::fs::File,
// Pid of the embed process
pub child_pid: nix::unistd::Pid,
// (width, height)
pub terminal_size: (usize, usize),
// Current colors
fg_color: Color,
bg_color: Color,
show_cursor: bool,
/* And others */
..
}And here’s the methods to control the terminal:
impl EmbedGrid {
pub fn set_terminal_size(&mut self, new_val: (usize, usize)) {
if new_val == self.terminal_size {
return;
}
self.terminal_size = new_val;
self.grid.resize(new_val.0, new_val.1, Cell::default());
self.grid.clear(Cell::default());
self.cursor = (0, 0);
let winsize = Winsize {
ws_row: <u16>::try_from(new_val.1).unwrap(),
ws_col: <u16>::try_from(new_val.0).unwrap(),
ws_xpixel: 0,
ws_ypixel: 0,
};
let master_fd = self.stdin.as_raw_fd();
/* This is the macrogenerated function we defined earlier, which calls
ioctl underneath */
unsafe { set_window_size(master_fd, &winsize).unwrap() };
/* SIGWINCH informs the process the tty size has changed. TUI apps must
handle this signal */
nix::sys::signal::kill(self.child_pid, nix::sys::signal::SIGWINCH).unwrap();
}
pub fn wake_up(&self) {
nix::sys::signal::kill(self.child_pid, nix::sys::signal::SIGCONT).unwrap();
}
pub fn stop(&self) {
/* we can emulate Ctrl-z and suspend the child process from meli */
nix::sys::signal::kill(self.child_pid, nix::sys::signal::SIGSTOP).unwrap();
}
pub fn process_byte(&mut self, byte: u8);
}In the process_byte method we take input from the child process and we perform actions depending on the control sequence state:
- if it’s
Normaland we get a byte that does not start a sequence, we output it on the grid. - if the state is in a sequence and we get a valid termination byte, we perform the sequence’s action
- if the state is in a sequence and we get a valid continuation byte, we update the state
- else, the child process sent malformed output or we haven’t implemented some sequence correctly (at this stage, probably the latter)
pub fn process_byte(&mut self, byte: u8) {
let EmbedGrid {
ref mut cursor,
ref terminal_size,
ref mut grid,
ref mut state,
ref mut stdin,
ref mut fg_color,
ref mut bg_color,
child_pid: _,
..
} = self;
macro_rules! increase_cursor_x {
() => {
if cursor.0 + 1 < terminal_size.0 {
cursor.0 += 1;
}
};
}
macro_rules! cursor_x {
() => {{
if cursor.0 >= terminal_size.0 {
cursor.0 = terminal_size.0.saturating_sub(1);
}
cursor.0
}};
}
macro_rules! cursor_y {
() => {
std::cmp::min(
cursor.1 + scroll_region.top,
terminal_size.1.saturating_sub(1),
)
};
}
macro_rules! cursor_val {
() => {
(cursor_x!(), cursor_y!())
};
}
let mut state = state;
match (byte, &mut state) {
(b'\x1b', State::Normal) => {
*state = State::ExpectingControlChar;
}
(b']', State::ExpectingControlChar) => {
let buf1 = Vec::new();
*state = State::Osc1(buf1);
}
(b'[', State::ExpectingControlChar) => {
*state = State::Csi;
}
/* 8<... */
(b'H', State::Csi) => {
/* move cursor to (1,1) */
*cursor = (0, 0);
*state = State::Normal;
}
/* 8<... */
(b'\r', State::Normal) => {
// carriage return x-> 0
cursor.0 = 0;
}
(b'\n', State::Normal) => {
// newline y -> y + 1
if cursor.1 + 1 < terminal_size.1 {
cursor.1 += 1;
}
}
/* replaced the actual ^G character here with '^' + 'G' for clarity: */
(b'^G', State::Normal) => {
// Visual bell ^G, ignoring
}
(0x08, State::Normal) => {
/* Backspace */
// x -> x - 1
if cursor.0 > 0 {
cursor.0 -= 1;
}
}
(c, State::Normal) => {
/* Character to be printed. */
/* We're UTF-8 bound, so check if byte starts a multi-byte
codepoint and keep state on this in order to get the complete
char, which will be 1-4 bytes long.
Check UTF-8 spec to see how it's defined.
*/
/* ...codepoint checks ...8< */
grid[cursor_val!()].set_ch(c);
grid[cursor_val!()].set_fg(*fg_color);
grid[cursor_val!()].set_bg(*bg_color);
increase_cursor_x!();
}
/* other various sequences: */
(b'H', State::Csi2(ref y, ref x)) => {
// Set Cursor Position [row;column] (default = [1,1])
let orig_x = unsafe { std::str::from_utf8_unchecked(x) }
.parse::<usize>()
.unwrap_or(1);
let orig_y = unsafe { std::str::from_utf8_unchecked(y) }
.parse::<usize>()
.unwrap_or(1);
if orig_x - 1 <= terminal_size.0 && orig_y - 1 <= terminal_size.1 {
cursor.0 = orig_x - 1;
cursor.1 = orig_y - 1;
} else {
eprintln!(
"[error] terminal_size = {:?}, cursor = {:?} but cursor set to [{},{}]",
terminal_size, cursor, orig_x, orig_y
);
}
*state = State::Normal;
}
(b'm', State::Csi1(ref buf1)) => {
// Set color
match buf1.as_slice() {
/* See next sequence for 38 and 48 special meanings */
b"30" => *fg_color = Color::Black,
b"31" => *fg_color = Color::Red,
b"32" => *fg_color = Color::Green,
b"33" => *fg_color = Color::Yellow,
b"34" => *fg_color = Color::Blue,
b"35" => *fg_color = Color::Magenta,
b"36" => *fg_color = Color::Cyan,
b"37" => *fg_color = Color::White,
b"39" => *fg_color = Color::Default,
b"40" => *bg_color = Color::Black,
b"41" => *bg_color = Color::Red,
b"42" => *bg_color = Color::Green,
b"43" => *bg_color = Color::Yellow,
b"44" => *bg_color = Color::Blue,
b"45" => *bg_color = Color::Magenta,
b"46" => *bg_color = Color::Cyan,
b"47" => *bg_color = Color::White,
b"49" => *bg_color = Color::Default,
_ => {}
}
grid[cursor_val!()].set_fg(*fg_color);
grid[cursor_val!()].set_bg(*bg_color);
*state = State::Normal;
}
(b'm', State::Csi3(ref buf1, ref buf2, ref buf3)) if buf1 == b"38" && buf2 == b"5" => {
/* ESC [ m 38 ; 5 ; fg_color_byte m */
/* Set only foreground color */
*fg_color = if let Ok(byte) =
u8::from_str_radix(unsafe { std::str::from_utf8_unchecked(buf3) }, 10)
{
Color::Byte(byte)
} else {
Color::Default
};
grid[cursor_val!()].set_fg(*fg_color);
*state = State::Normal;
}
(b'm', State::Csi3(ref buf1, ref buf2, ref buf3)) if buf1 == b"48" && buf2 == b"5" => {
/* ESC [ m 48 ; 5 ; fg_color_byte m */
/* Set only background color */
*bg_color = if let Ok(byte) =
u8::from_str_radix(unsafe { std::str::from_utf8_unchecked(buf3) }, 10)
{
Color::Byte(byte)
} else {
Color::Default
};
grid[cursor_val!()].set_bg(*bg_color);
*state = State::Normal;
}
(b'D', State::Csi1(buf)) => {
// ESC[{buf}D CSI Cursor Backward {buf} Times
let offset = unsafe { std::str::from_utf8_unchecked(buf) }
.parse::<usize>()
.unwrap();
cursor.0 = cursor.0.saturating_sub(offset);
*state = State::Normal;
}
/* and others */
}
}A bit of UI glue code later: