Arduino strtok: splitting text and parsing serial commands
Or see every lesson.
- 455parts on the bench
- 43boards running now
- 1.00xreal time, on every board
strtok() splits a char array into pieces at the characters you name, such as commas or spaces. The first call takes the array, every later call takes NULL and carries on where the last one stopped, and it returns NULL when there is nothing left. It works on plain C text rather than a String, so it uses little memory and suits the short lines a sensor or the serial monitor sends.
The circuit above is an Uno that echoes what you type, one byte at a time. Typed "on" and Enter, it printed "you typed: o (0x6F)", "you typed: n (0x6E)" and "you typed: <newline>". Bytes arrive one by one, so a sketch gathers them into a line first, and only then splits it.
Below is how to split comma separated values into numbers, what strtok does to your array, why a String needs a copy, a command parser that switches LEDs, and strtok_r for a split inside a split. To split a String with indexOf() and substring() instead, see the String page.
Want the step-by-step version? The lesson "Taking commands" walks through this with checkpoints.
Open the lessonSplit comma separated values into an array
Call strtok(line, ",") once for the first piece, then strtok(NULL, ",") in a loop until it returns NULL, converting each piece with atof() for decimals or atoi() for whole numbers. The sketch below printed "3 values", then "21.50", "48.00" and "1013.00".
Stop at the size of your array, as the count < 3 does: a line with more commas than you expected must not write past the end.
The last line shows what strtok leaves behind. It printed "line is now: 21.5". strtok writes a zero over each comma it finds, so the array now ends after the first piece. Copy the line first if you need it again.
void setup() {
Serial.begin(9600);
char line[] = "21.5,48,1013";
float values[3];
int count = 0;
char *piece = strtok(line, ",");
while (piece != NULL && count < 3) {
values[count] = atof(piece);
count++;
piece = strtok(NULL, ",");
}
Serial.print(count);
Serial.println(" values");
for (int i = 0; i < count; i++) {
Serial.println(values[i]);
}
Serial.print("line is now: ");
Serial.println(line);
}
void loop() {
}Empty fields disappear
strtok treats a run of commas as one, so an empty field is skipped rather than returned as empty. The same sketch with "5,,7" printed "2 values", 5.00 and 7.00: the 7 moved into the second place. When a field can be blank, check the count, or send a 0 instead of nothing.
strtok needs a char array, not a String
strtok changes the text it splits, so it takes a char *. Giving it a String’s text with strtok(line.c_str(), ",") stopped the build with no matching function for call to 'strtok', and the note says 1st argument ('const char *') would lose const qualifier: c_str() is read only. The Arduino IDE’s GCC says invalid conversion from 'const char*' to 'char*'.
Copy the String into a char array first with toCharArray(). The sketch below printed "21.5" and "48". A quoted string written straight into the call, strtok("21.5,48", ","), only got a warning, ISO C++11 does not allow conversion from string literal to 'char *', and GCC’s is ISO C++ forbids converting a string constant to 'char*'. It ran here and printed 21.5, but do not rely on it: text in quotes is meant to stay as it is, and strtok writes into it.
void setup() {
Serial.begin(9600);
String line = "21.5,48";
char buf[32];
line.toCharArray(buf, sizeof(buf));
char *piece = strtok(buf, ",");
while (piece != NULL) {
Serial.println(piece);
piece = strtok(NULL, ",");
}
}
void loop() {
}A serial command parser
The sketch below collects bytes into a 32 byte buffer until a newline, then splits the line at spaces into a command and two arguments and compares the command with strcmp(). It understands "led 13 on", "led 12 off" and "blink 11 3". Anything else gets an answer too, so a typo is never silent.
We typed five lines: led 13 on, led 12 on, led 13 off, blink 11 3 and hello. It printed "pin 13 on", "pin 12 on", "pin 13 off", "blinked 3 times" and "unknown: hello", and left the LED on pin 12 lit. The len < sizeof(buf) - 1 check drops anything past 31 characters rather than writing past the buffer, and the \r test makes it work whichever line ending the monitor sends. To accept commas as well as spaces between the words, give strtok " ," as its separators instead of " ".
char buf[32];
byte len = 0;
void setup() {
Serial.begin(9600);
pinMode(13, OUTPUT);
pinMode(12, OUTPUT);
pinMode(11, OUTPUT);
Serial.println("ready");
}
void runCommand(char *line) {
char *cmd = strtok(line, " ");
char *arg1 = strtok(NULL, " ");
char *arg2 = strtok(NULL, " ");
if (cmd == NULL) return;
if (strcmp(cmd, "led") == 0 && arg1 && arg2) {
int pin = atoi(arg1);
digitalWrite(pin, strcmp(arg2, "on") == 0 ? HIGH : LOW);
Serial.print("pin ");
Serial.print(pin);
Serial.print(" ");
Serial.println(arg2);
} else if (strcmp(cmd, "blink") == 0 && arg1 && arg2) {
int pin = atoi(arg1);
int times = atoi(arg2);
for (int i = 0; i < times; i++) {
digitalWrite(pin, HIGH);
delay(100);
digitalWrite(pin, LOW);
delay(100);
}
Serial.print("blinked ");
Serial.print(times);
Serial.println(" times");
} else {
Serial.print("unknown: ");
Serial.println(cmd);
}
}
void loop() {
while (Serial.available() > 0) {
char c = Serial.read();
if (c == '\r') continue;
if (c == '\n') {
buf[len] = '\0';
runCommand(buf);
len = 0;
} else if (len < sizeof(buf) - 1) {
buf[len++] = c;
}
}
}A split inside a split: strtok_r
strtok remembers its place in one hidden variable, so it can only work through one line at a time. Splitting "temp=21;hum=48;lux=310" at semicolons, and each pair at the equals sign with strtok as well, printed "temp -> 21" and stopped: the inner calls lost the outer one’s place.
strtok_r takes the place as a third argument, so each level keeps its own. The version below printed "temp -> 21", "hum -> 48" and "lux -> 310". Use strtok_r whenever a function that splits text can be called from inside another split.
void setup() {
Serial.begin(9600);
char line[] = "temp=21;hum=48;lux=310";
char *outer;
char *pair = strtok_r(line, ";", &outer);
while (pair != NULL) {
char *inner;
char *key = strtok_r(pair, "=", &inner);
char *value = strtok_r(NULL, "=", &inner);
Serial.print(key);
Serial.print(" -> ");
Serial.println(atoi(value));
pair = strtok_r(NULL, ";", &outer);
}
}
void loop() {
}Questions
How does strtok work in Arduino?
The first call, strtok(line, ","), returns the first piece of a char array; each later call, strtok(NULL, ","), returns the next, and NULL means there are no more. It replaces each separator in the array with a zero as it goes.
Can I use strtok with a String?
Not directly: c_str() is read only and strtok writes into the text, so the build stops. Copy the String into a char array with toCharArray() and split that.
Why does strtok skip my empty field?
It treats two separators in a row as one, so "5,,7" gives two pieces, not three. Check how many pieces came back, or send a 0 for a missing value.
How do I parse commands from the serial monitor?
Collect bytes into a char array until a newline, end it with a zero, split it at spaces with strtok, and compare the first piece with strcmp(). The parser on this page does it for LED commands.
What is the difference between strtok and strtok_r?
strtok keeps its place in one hidden variable, so two splits at once interfere. strtok_r takes the place as an argument, so a split inside a split works.
Keep going
Where the facts on this page come from
Build this for real
Open the editor, change a value and watch the number move with it. Nothing to install, and no account needed.