Text processing

Slice, filter, count and transform log files, CSV and JSON on the command line. awk, sed, jq, sort, uniq and cut are tools every admin needs daily.

24 one-liners: 22 harmless, 2 caution, 0 destructive

jq 6 json 6 awk 5 columns 5 sort 3 lines 3 filter 3 sed 3 formatting 2 csv 2 wordpress 2 merge 2 excerpt 2 configuration 2

For the websites on your server

Free SEO and PageSpeed check

GENLOC.SEO analyzes any domain for load time, Core Web Vitals and on-page issues and tells you in plain words what is going on. Available in 11 languages.

by GENLOC.NETWORK, the team behind myline.de

Count the most frequent lines in a file

$sort list.txt | uniq -c | sort -rn | head -n 20

harmless

Count WordPress plugins by status

$wp plugin list --format=json --path=/var/www/vhosts/example.com/httpdocs | jq -r 'group_by(.status)[] | "\(.[0].status): \(length)"'

harmless

Extract a single field from JSON with jq

$jq -r '.require.php' composer.json

harmless

Extract specific fields with cut

$cut -d':' -f1,7 /etc/passwd

harmless

Filter a JSON array by a field value

$jq '[.[] | select(.status == "active")]' plugins.json

harmless

Filter CSV rows by the value in a column

$awk -F';' -v c=3 -v w='active' 'NR == 1 || $c == w' export.csv

harmless

Get the latest WordPress version from the API

$curl -s https://api.wordpress.org/core/version-check/1.7/ | jq -r '.offers[0].version'

harmless

Join two files on a common field

$join -t ';' <(sort -t ';' -k1,1 domains-customers.csv) <(sort -t ';' -k1,1 domains-expiry.csv)

harmless

Number the lines of a file

$nl -ba /var/www/vhosts/example.com/httpdocs/.htaccess

harmless

Pretty-print JSON with indentation and colors

$jq . composer.json

harmless

Print a specific range of lines from a file

$sed -n '100,120p;120q' /var/www/vhosts/example.com/httpdocs/wp-config.php

harmless

Print lines between two patterns

$sed -n '/BEGIN CERTIFICATE/,/END CERTIFICATE/p' fullchain.pem

harmless

Print lines where a column value exceeds a threshold

$awk -v c=10 -v min=5000000 '$c + 0 > min + 0' /var/www/vhosts/system/example.com/logs/access_ssl_log

harmless

Put two files side by side line by line

$paste -d ';' domains.txt ips.txt

harmless

Remove duplicate lines without changing the order

$awk '!seen[$0]++' list.txt

harmless

Replace text in a file with sed and keep a backup

$sed -i.bak 's|memory_limit = 128M|memory_limit = 256M|g' php.ini

caution

Show a config file without comments and blank lines

$grep -Ev '^\s*(#|;|$)' /etc/php/8.2/fpm/php.ini

harmless

Sum the numbers in a column with awk

$awk -v c=10 '{s += $c} END {print s}' /var/www/vhosts/system/example.com/logs/access_ssl_log

harmless

All Text processing one-liners

Align output as a table with column

Calculate the average, minimum and maximum of a column

Compare two lists: what is missing from the second

Convert a JSON array to CSV with a header row

Convert text to lowercase or uppercase

Count and sort lines across many files

Count the most frequent lines in a file

Count WordPress plugins by status

Extract a single field from JSON with jq

Extract specific fields with cut

Filter a JSON array by a field value

Filter CSV rows by the value in a column

Get the latest WordPress version from the API

Join two files on a common field

Number the lines of a file

Pretty-print JSON with indentation and colors

Print a specific range of lines from a file

Print lines between two patterns

Print lines where a column value exceeds a threshold

Put two files side by side line by line

Remove duplicate lines without changing the order

Replace text in a file with sed and keep a backup

Show a config file without comments and blank lines

Sum the numbers in a column with awk