Remove and replace patterns using sed

Tags: editor, linux, sed
Publish Date: 2016-04-26

sed is a stream editor, which transforms text from an input stream and sends it to output stream.
It is often used to (globally) replace or delete patterns, often used by sys-admins to change configuration files.
 

Unless specified explicitly, it won't change the contents of your file. For this reason, it is a great tool to preview changes on your screen before writing them to file.


Let's take the /etc/logrotate.conf file as an example. In case you plan to follow along with this example, please create a copy of the logrotate.conf to a separate location first and try it out from there instead of /etc !

# see "man logrotate" for details
# rotate log files weekly
weekly

# use the syslog group by default, since this is the owning group
# of /var/log/syslog.
su root syslog

# keep 4 weeks worth of backlogs
rotate 4

# create new (empty) log files after rotating old ones
create

# uncomment this if you want your log files compressed
#compress

# packages drop log rotation information into this directory
include /etc/logrotate.d

# no packages own wtmp, or btmp -- we'll rotate them here
/var/log/wtmp {
    missingok
    monthly
    create 0664 root utmp
    rotate 1
}

/var/log/btmp {
    missingok
    monthly
    create 0660 root utmp
    rotate 1
}

# system-specific logs may be configured here

Deleting Patterns

If we want to remove all commented lines in this file, it means we should delete all lines that starts with a #, preceded by zero or more whitespaces. The pattern that we need for sed would be:

/^\s*#/d

# Whereby:
# ^  => Anchor for start of line
# \s => Escape sequence for any whitespace character
# *  => Quantifier for zero or more

# The letter d at the end of the pattern is a sed-command for delete

By running:

sed /^\s*#/d logrotate.conf

It would send the text without commented lines to the STDOUT, without writing to the file.

If you do want to change the file, the -i (--in-place) option has to be provided to sed.
The -i option can also take a suffix, by doing so, it would create a backup of the original file and adds the suffix to the backup's filename. Example:

sed -i.bak /^\s*#/d logrotate.conf

This will create a file called logrotate.conf.bak with the original content of logrotate.conf (with the commented lines), then writes the changes (removing the commented lines) to logrotate.conf.
The result of logrotate.conf would be:

weekly

su root syslog

rotate 4

create


include /etc/logrotate.d

/var/log/wtmp {
    missingok
    monthly
    create 0664 root utmp
    rotate 1
}

/var/log/btmp {
    missingok
    monthly
    create 0660 root utmp
    rotate 1
}

Please note that when no suffix is provided to -i, it would write changes to the file directly without creating a backup.

Change patterns

If we, for whatever reason, would like to change all log-rotations that are currently set to monthly to weekly, we could also use sed to perform a quick find-and-replace.
For that, we need the s sed-command:


s/monthly/weekly/g

# Whereby:
# s => sed-command to search for a given RegExp (monthly) and substitute it with a replacement (weekly).
# g => RegExp Modifier, to replace globally instead of first occurrence per line.

# In our case, the word "monthly" only occurs once per line, so the g modifier is optional.

By performing:


sed s/monthly/weekly/g logrotate.conf

It would change all occurrences of the word monthly to weekly, without writing to file
Again, if you wish to write to file, provide the -i option as mentioned earlier.