The command prompt reads one string and clap reads an argv, so a line has to be tokenised before anything can parse it: `comment 1b78a24 the offset was never passed` split on whitespace is five arguments and `comment` takes two. One pure function over the line and the cursor, in a module of its own rather than in tui.rs. The rules are the shell's, because that is what the prompt promises. Two answers a shell's own splitter does not owe anybody come out of the same scan: what a half-typed line is still inside, as a state to report rather than an error to raise, and which argument the cursor is in with whether it begins one. Nothing calls it yet. The prompt that will is 3686dd6, and the dead_code allow on the module goes when it lands. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
454 lines
18 KiB
Rust
454 lines
18 KiB
Rust
//! Splitting a typed line into the arguments a parser wants.
|
|
//!
|
|
//! The prompt on the board takes one string and clap takes an argv, so something has to
|
|
//! turn `comment 1b78a24 the offset was never passed` into a verb, a ticket and one body
|
|
//! with spaces in it. Splitting on whitespace hands clap five arguments where `comment`
|
|
//! takes two, which is not a grammar error to report but a tokenising step nobody ran.
|
|
//!
|
|
//! The rules are the shell's, because that is what the prompt promises: a line is spelled
|
|
//! there exactly as it is spelled in the shell. Runs of whitespace separate arguments,
|
|
//! single and double quotes group one, a backslash escapes the next character except
|
|
//! inside single quotes where it is a backslash, and an explicitly empty `''` is an
|
|
//! argument that exists and is empty — which is what `--title ""` being a usage error
|
|
//! rather than a silent clear depends on. A leading hyphen is not special here: whether
|
|
//! `--foo` is a flag or the text of a criterion is clap's question, and the prose
|
|
//! arguments already accept one.
|
|
//!
|
|
//! Two things a shell's own splitter does not have to answer, and both are why this is
|
|
//! written rather than pulled in:
|
|
//!
|
|
//! - **A line being typed is half finished.** An unterminated quote is a state to report,
|
|
//! not an error to raise, because the prompt has to keep taking characters and refuse
|
|
//! only enter. See [`Unclosed`].
|
|
//! - **Completion needs to know where the cursor is.** Which argument it sits in, counting
|
|
//! from the verb, and whether nothing of that argument lies before it. Both fall out of
|
|
//! the scan for free and neither is reachable afterwards from the arguments alone.
|
|
//!
|
|
//! One pure function over the line and the cursor, and nothing here knows about `Board`.
|
|
|
|
/// What a line is still inside when it ends.
|
|
///
|
|
/// A half-typed line is not a line that failed: `comment 1b78a24 "the offset` is a
|
|
/// sentence somebody is in the middle of, and the arguments before the quote are already
|
|
/// right. So this is a state the caller reports, and the reason enter is refused while it
|
|
/// is anything but [`Unclosed::Closed`] — the arguments up to that point are still handed
|
|
/// back, because that is what a completion or a hint has to work from.
|
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
|
pub enum Unclosed {
|
|
/// Nothing. The arguments are whole and the line can go to a parser.
|
|
Closed,
|
|
/// A `'` with no `'` after it.
|
|
Single,
|
|
/// A `"` with no `"` after it.
|
|
Double,
|
|
/// A trailing `\`, which has nothing yet to escape.
|
|
///
|
|
/// Not a quote, and reported the same way for the same reason: the character it will
|
|
/// escape is the next one typed. It wins over an open quote, being the inner of the
|
|
/// two, and either answer refuses the line.
|
|
Escape,
|
|
}
|
|
|
|
/// Which quote the scan is inside.
|
|
///
|
|
/// Private, because what a caller needs is [`Unclosed`], which also carries the state no
|
|
/// quote can. Two public spellings of one fact is how they come to disagree.
|
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
|
enum Quote {
|
|
Single,
|
|
Double,
|
|
}
|
|
|
|
/// One scan of a line: its arguments, where the cursor sits among them, and what the line
|
|
/// is still inside.
|
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
|
pub struct Split {
|
|
/// The arguments, in order, with the quotes and escapes taken off. The binary's own
|
|
/// name is not among them: prepending it is the caller's, since only the caller knows
|
|
/// what it is calling.
|
|
pub args: Vec<String>,
|
|
/// Which argument the cursor is in, counting from the verb at 0. Equal to
|
|
/// `args.len()` when the cursor is past the last one, which is the index an argument
|
|
/// typed next would take.
|
|
pub cursor_arg: usize,
|
|
/// Whether nothing of `cursor_arg` lies before the cursor, because the line up to it
|
|
/// ends in a separator or has nothing in it at all. That is the difference between
|
|
/// completing an argument and starting one.
|
|
pub at_new_arg: bool,
|
|
/// What the line is still inside, and therefore whether it can be run at all.
|
|
pub unclosed: Unclosed,
|
|
}
|
|
|
|
/// Split `line` into arguments, reading the cursor at `cursor` as a byte offset into it.
|
|
///
|
|
/// Bytes because that is what the prompt's own cursor is: a `String` inserts and removes
|
|
/// at byte indices. A cursor past the end of the line reads as the end, and one inside a
|
|
/// multi-byte character reads as being after that character, so neither can panic and
|
|
/// neither needs the caller to have clamped it first.
|
|
pub fn split_line(line: &str, cursor: usize) -> Split {
|
|
let mut args: Vec<String> = Vec::new();
|
|
// `Some("")` is an argument that has been opened and is still empty, which is what
|
|
// tells `''` from nothing at all: a bare `take()` cannot make that distinction.
|
|
let mut arg: Option<String> = None;
|
|
let mut quote: Option<Quote> = None;
|
|
let mut escape = false;
|
|
// The cursor's answer, taken at the character it falls on and never revised.
|
|
let mut at: Option<(usize, bool)> = None;
|
|
|
|
for (offset, c) in line.char_indices() {
|
|
if at.is_none() && offset >= cursor {
|
|
at = Some((args.len(), arg.is_none()));
|
|
}
|
|
|
|
if escape {
|
|
arg.get_or_insert_default().push(c);
|
|
escape = false;
|
|
continue;
|
|
}
|
|
|
|
match (quote, c) {
|
|
// The shell's exception, and the reason a criterion reading `a\b` survives
|
|
// being typed as `'a\b'`: single quotes are literal, backslash included.
|
|
(Some(Quote::Single), '\\') => arg.get_or_insert_default().push(c),
|
|
(_, '\\') => escape = true,
|
|
(None, '\'') => {
|
|
quote = Some(Quote::Single);
|
|
arg.get_or_insert_default();
|
|
}
|
|
(None, '"') => {
|
|
quote = Some(Quote::Double);
|
|
arg.get_or_insert_default();
|
|
}
|
|
(Some(Quote::Single), '\'') | (Some(Quote::Double), '"') => quote = None,
|
|
// A run of separators closes the open argument once and then finds none.
|
|
(None, ws) if ws.is_whitespace() => {
|
|
if let Some(done) = arg.take() {
|
|
args.push(done);
|
|
}
|
|
}
|
|
_ => arg.get_or_insert_default().push(c),
|
|
}
|
|
}
|
|
|
|
// Before the last argument is pushed, so a cursor at the end of the line is still in
|
|
// the argument it is typing rather than one past it.
|
|
let (cursor_arg, at_new_arg) = at.unwrap_or((args.len(), arg.is_none()));
|
|
if let Some(last) = arg {
|
|
args.push(last);
|
|
}
|
|
|
|
Split {
|
|
args,
|
|
cursor_arg,
|
|
at_new_arg,
|
|
unclosed: match (escape, quote) {
|
|
(true, _) => Unclosed::Escape,
|
|
(false, Some(Quote::Single)) => Unclosed::Single,
|
|
(false, Some(Quote::Double)) => Unclosed::Double,
|
|
(false, None) => Unclosed::Closed,
|
|
},
|
|
}
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
/// One line and everything the scan is supposed to say about it.
|
|
///
|
|
/// The defaults are the empty line's own answers, so a row states the rule it is about
|
|
/// and stays quiet about the rest.
|
|
struct Case {
|
|
line: &'static str,
|
|
/// Where the cursor is, or the end of the line, which is where it sits while a
|
|
/// line is being typed. Only the cursor rows move it.
|
|
cursor: Option<usize>,
|
|
args: &'static [&'static str],
|
|
cursor_arg: usize,
|
|
at_new_arg: bool,
|
|
unclosed: Unclosed,
|
|
}
|
|
|
|
impl Default for Case {
|
|
fn default() -> Self {
|
|
Case {
|
|
line: "",
|
|
cursor: None,
|
|
args: &[],
|
|
cursor_arg: 0,
|
|
at_new_arg: true,
|
|
unclosed: Unclosed::Closed,
|
|
}
|
|
}
|
|
}
|
|
|
|
/// Every field of every row, each named in its own failure: a table that reports only
|
|
/// the first mismatch makes the next run tell a different story than this one.
|
|
fn check(cases: &[Case]) {
|
|
for case in cases {
|
|
let cursor = case.cursor.unwrap_or(case.line.len());
|
|
let split = split_line(case.line, cursor);
|
|
let args: Vec<&str> = split.args.iter().map(String::as_str).collect();
|
|
let at = format!("{:?} with the cursor at {cursor}", case.line);
|
|
|
|
assert_eq!(args, case.args, "arguments of {at}");
|
|
assert_eq!(split.cursor_arg, case.cursor_arg, "cursor argument of {at}");
|
|
assert_eq!(split.at_new_arg, case.at_new_arg, "new argument of {at}");
|
|
assert_eq!(split.unclosed, case.unclosed, "unclosed state of {at}");
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn a_line_splits_into_arguments_the_way_a_shell_splits_one() {
|
|
check(&[
|
|
// Nothing typed yet, which is the state the prompt opens in.
|
|
Case::default(),
|
|
Case {
|
|
line: "tags",
|
|
args: &["tags"],
|
|
at_new_arg: false,
|
|
..Case::default()
|
|
},
|
|
Case {
|
|
line: "mv 1b78a24 --status committed",
|
|
args: &["mv", "1b78a24", "--status", "committed"],
|
|
cursor_arg: 3,
|
|
at_new_arg: false,
|
|
..Case::default()
|
|
},
|
|
// Runs of separators collapse, at either end and in the middle, and a tab is
|
|
// one of them.
|
|
Case {
|
|
line: " mv 1b78a24 ",
|
|
args: &["mv", "1b78a24"],
|
|
cursor_arg: 2,
|
|
..Case::default()
|
|
},
|
|
Case {
|
|
line: "mv\t1b78a24",
|
|
args: &["mv", "1b78a24"],
|
|
cursor_arg: 1,
|
|
at_new_arg: false,
|
|
..Case::default()
|
|
},
|
|
// What the ticket for this was written about: one body, spaces and all.
|
|
Case {
|
|
line: "comment 1b78a24 \"the offset was never passed\"",
|
|
args: &["comment", "1b78a24", "the offset was never passed"],
|
|
cursor_arg: 2,
|
|
at_new_arg: false,
|
|
..Case::default()
|
|
},
|
|
// Single quotes group the same, and a run of spaces inside one is not a
|
|
// separator to collapse.
|
|
Case {
|
|
line: "comment 1b78a24 'two spaces'",
|
|
args: &["comment", "1b78a24", "two spaces"],
|
|
cursor_arg: 2,
|
|
at_new_arg: false,
|
|
..Case::default()
|
|
},
|
|
// A quote in the middle of an argument groups part of it, which is how a
|
|
// `--flag=value` carrying a space is typed.
|
|
Case {
|
|
line: "set 1b78a24 --title=\"a b\"",
|
|
args: &["set", "1b78a24", "--title=a b"],
|
|
cursor_arg: 2,
|
|
at_new_arg: false,
|
|
..Case::default()
|
|
},
|
|
// A backslash escapes the next character: a separator, a quote, itself.
|
|
Case {
|
|
line: "comment 1b78a24 a\\ b",
|
|
args: &["comment", "1b78a24", "a b"],
|
|
cursor_arg: 2,
|
|
at_new_arg: false,
|
|
..Case::default()
|
|
},
|
|
Case {
|
|
line: "comment 1b78a24 \"she said \\\"no\\\"\"",
|
|
args: &["comment", "1b78a24", "she said \"no\""],
|
|
cursor_arg: 2,
|
|
at_new_arg: false,
|
|
..Case::default()
|
|
},
|
|
Case {
|
|
line: "comment 1b78a24 a\\\\b",
|
|
args: &["comment", "1b78a24", "a\\b"],
|
|
cursor_arg: 2,
|
|
at_new_arg: false,
|
|
..Case::default()
|
|
},
|
|
// Except inside single quotes, where it is a character like any other.
|
|
Case {
|
|
line: "comment 1b78a24 'a\\b'",
|
|
args: &["comment", "1b78a24", "a\\b"],
|
|
cursor_arg: 2,
|
|
at_new_arg: false,
|
|
..Case::default()
|
|
},
|
|
// A leading hyphen is clap's business, not the splitter's.
|
|
Case {
|
|
line: "add --title --print is supported",
|
|
args: &["add", "--title", "--print", "is", "supported"],
|
|
cursor_arg: 4,
|
|
at_new_arg: false,
|
|
..Case::default()
|
|
},
|
|
// An empty quoted value is an argument that exists and is empty. Losing it
|
|
// here would turn a usage error into a silent clear.
|
|
Case {
|
|
line: "set 1b78a24 --title \"\"",
|
|
args: &["set", "1b78a24", "--title", ""],
|
|
cursor_arg: 3,
|
|
at_new_arg: false,
|
|
..Case::default()
|
|
},
|
|
Case {
|
|
line: "''",
|
|
args: &[""],
|
|
at_new_arg: false,
|
|
..Case::default()
|
|
},
|
|
]);
|
|
}
|
|
|
|
#[test]
|
|
fn a_half_typed_line_says_what_it_is_still_inside_and_hands_back_the_rest() {
|
|
check(&[
|
|
// The arguments before the open quote are already right, and the open one is
|
|
// as far as it has been typed.
|
|
Case {
|
|
line: "comment 1b78a24 \"the offset",
|
|
args: &["comment", "1b78a24", "the offset"],
|
|
cursor_arg: 2,
|
|
at_new_arg: false,
|
|
unclosed: Unclosed::Double,
|
|
..Case::default()
|
|
},
|
|
Case {
|
|
line: "comment 1b78a24 'the offset",
|
|
args: &["comment", "1b78a24", "the offset"],
|
|
cursor_arg: 2,
|
|
at_new_arg: false,
|
|
unclosed: Unclosed::Single,
|
|
..Case::default()
|
|
},
|
|
// A quote just opened is an argument that exists and is empty, the same shape
|
|
// a closed `''` leaves.
|
|
Case {
|
|
line: "comment 1b78a24 \"",
|
|
args: &["comment", "1b78a24", ""],
|
|
cursor_arg: 2,
|
|
at_new_arg: false,
|
|
unclosed: Unclosed::Double,
|
|
..Case::default()
|
|
},
|
|
// A trailing backslash has nothing yet to escape. It opened no argument, so
|
|
// the cursor is starting one.
|
|
Case {
|
|
line: "mv 1b78a24 \\",
|
|
args: &["mv", "1b78a24"],
|
|
cursor_arg: 2,
|
|
unclosed: Unclosed::Escape,
|
|
..Case::default()
|
|
},
|
|
// Inside a quote the escape is the inner of the two, and it is what the line
|
|
// is waiting on.
|
|
Case {
|
|
line: "comment 1b78a24 \"a\\",
|
|
args: &["comment", "1b78a24", "a"],
|
|
cursor_arg: 2,
|
|
at_new_arg: false,
|
|
unclosed: Unclosed::Escape,
|
|
..Case::default()
|
|
},
|
|
// A closed quote reopened is open again: the state is where the scan ended,
|
|
// not how many quotes were typed.
|
|
Case {
|
|
line: "comment 1b78a24 'a' 'b",
|
|
args: &["comment", "1b78a24", "a", "b"],
|
|
cursor_arg: 3,
|
|
at_new_arg: false,
|
|
unclosed: Unclosed::Single,
|
|
..Case::default()
|
|
},
|
|
]);
|
|
}
|
|
|
|
#[test]
|
|
fn the_split_says_which_argument_the_cursor_is_in_and_whether_it_begins_one() {
|
|
check(&[
|
|
// The line is split whole whatever the cursor is doing: only the two cursor
|
|
// answers move.
|
|
Case {
|
|
line: "mv 1b78a24",
|
|
cursor: Some(0),
|
|
args: &["mv", "1b78a24"],
|
|
..Case::default()
|
|
},
|
|
Case {
|
|
line: "mv 1b78a24",
|
|
cursor: Some(1),
|
|
args: &["mv", "1b78a24"],
|
|
at_new_arg: false,
|
|
..Case::default()
|
|
},
|
|
// At the end of the verb, still in the verb: the cursor is completing it.
|
|
Case {
|
|
line: "mv 1b78a24",
|
|
cursor: Some(2),
|
|
args: &["mv", "1b78a24"],
|
|
at_new_arg: false,
|
|
..Case::default()
|
|
},
|
|
// Past the separator, so the argument at 1 has nothing before the cursor even
|
|
// though the line goes on to hold one.
|
|
Case {
|
|
line: "mv 1b78a24",
|
|
cursor: Some(3),
|
|
args: &["mv", "1b78a24"],
|
|
cursor_arg: 1,
|
|
..Case::default()
|
|
},
|
|
// A cursor past the end of the line reads as the end of it.
|
|
Case {
|
|
line: "mv",
|
|
cursor: Some(99),
|
|
args: &["mv"],
|
|
at_new_arg: false,
|
|
..Case::default()
|
|
},
|
|
// Inside a quoted argument the cursor is in that argument, not in a fourth
|
|
// one the quotes might look like they opened.
|
|
Case {
|
|
line: "comment 1b78a24 \"the offset\"",
|
|
cursor: Some(20),
|
|
args: &["comment", "1b78a24", "the offset"],
|
|
cursor_arg: 2,
|
|
at_new_arg: false,
|
|
..Case::default()
|
|
},
|
|
// Byte offsets, so a multi-byte character before the cursor counts for its
|
|
// width and not for one: 8 is where `é` starts and 9 is inside it.
|
|
Case {
|
|
line: "comment é",
|
|
cursor: Some(8),
|
|
args: &["comment", "é"],
|
|
cursor_arg: 1,
|
|
..Case::default()
|
|
},
|
|
// A cursor inside a character reads as being after it, which is the answer
|
|
// that cannot claim an argument has not started when it has.
|
|
Case {
|
|
line: "comment é",
|
|
cursor: Some(9),
|
|
args: &["comment", "é"],
|
|
cursor_arg: 1,
|
|
at_new_arg: false,
|
|
..Case::default()
|
|
},
|
|
]);
|
|
}
|
|
}
|