Remove and replace patterns using sed

Tags: editor, linux, sed
Publish Date: 2016-04-26

sed is a stream editor, which transforms text from an input stream and sends it to output stream.
It is often used to (globally) replace or delete patterns, often used by sys-admins to change configuration files.
 

Unless specified explicitly, it won't change the contents of your file. For this reason, it is a great tool to preview changes on your screen before writing them to file.


Let's take the /etc/logrotate.conf file as an example. In case you plan to follow along with this example, please create a copy of the logrotate.conf to a separate location first and try it out from there instead of /etc !

# see "man logrotate" for details
# rotate log files weekly
weekly

# use the syslog group by default, since this is the owning group
# of /var/log/syslog.
su root syslog

# keep 4 weeks worth of backlogs
rotate 4

# create new (empty) log files after rotating old ones
create

# uncomment this if you want your log files compressed
#compress

# packages drop log rotation information into this directory
include /etc/logrotate.d

# no packages own wtmp, or btmp -- we'll rotate them here
/var/log/wtmp {
    missingok
    monthly
    create 0664 root utmp
    rotate 1
}

/var/log/btmp {
    missingok
    monthly
    create 0660 root utmp
    rotate 1
}

# system-specific logs may be configured here

Deleting Patterns

If we want to remove all commented lines in this file, it means we should delete all lines that starts with a #, preceded by zero or more whitespaces. The pattern that we need for sed would be:

/^\s*#/d

# Whereby:
# ^  => Anchor for start of line
# \s => Escape sequence for any whitespace character
# *  => Quantifier for zero or more

# The letter d at the end of the pattern is a sed-command for delete

By running:

sed /^\s*#/d logrotate.conf

It would send the text without commented lines to the STDOUT, without writing to the file.

If you do want to change the file, the -i (--in-place) option has to be provided to sed.
The -i option can also take a suffix, by doing so, it would create a backup of the original file and adds the suffix to the backup's filename. Example:

sed -i.bak /^\s*#/d logrotate.conf

This will create a file called logrotate.conf.bak with the original content of logrotate.conf (with the commented lines), then writes the changes (removing the commented lines) to logrotate.conf.
The result of logrotate.conf would be:

weekly

su root syslog

rotate 4

create


include /etc/logrotate.d

/var/log/wtmp {
    missingok
    monthly
    create 0664 root utmp
    rotate 1
}

/var/log/btmp {
    missingok
    monthly
    create 0660 root utmp
    rotate 1
}

Please note that when no suffix is provided to -i, it would write changes to the file directly without creating a backup.

Change patterns

If we, for whatever reason, would like to change all log-rotations that are currently set to monthly to weekly, we could also use sed to perform a quick find-and-replace.
For that, we need the s sed-command:


s/monthly/weekly/g

# Whereby:
# s => sed-command to search for a given RegExp (monthly) and substitute it with a replacement (weekly).
# g => RegExp Modifier, to replace globally instead of first occurrence per line.

# In our case, the word "monthly" only occurs once per line, so the g modifier is optional.

By performing:


sed s/monthly/weekly/g logrotate.conf

It would change all occurrences of the word monthly to weekly, without writing to file
Again, if you wish to write to file, provide the -i option as mentioned earlier.

Map method in Javascript and Ruby

Tags: javascript, ruby
Publish Date: 2014-11-06

The map method in Ruby (defined in the Enumerable module) is one of the most commonly used methods among Ruby developers. However, in the Javascript world, it only became cross-browser compatible since IE9.

In Ruby, #map allows you to run a block of code on every element in an "enumerable object" and returns the results in an Array.
By an "enumerable object", I mean Ruby classes that includes the Enumerable module, such as Array, Hash and Set.

In the following example, we have a 2-dimensional array, in which every sub-array contains the first name and last name as two separate array elements. By providing a block of code to the map method, we can return the full names as a string and store them in an array.

  split_names = [ ["Carmelo", "Anthony"], 
                  ["Rajon", "Rondo"], 
                  ["Tim", "Duncan"], 
                  ["Derrick", "Rose"] ]

  names = split_names.map{ |elem| "#{elem[0]} #{elem[1]}" }

  names #=> ["Carmelo Anthony", "Rajon Rondo", "Tim Duncan", "Derrick Rose"]

In Javascript, the same can be done with the map method, which takes an annonymous function:

  var split_names = [ ["Carmelo", "Anthony"], 
                      ["Rajon", "Rondo"], 
                      ["Tim", "Duncan"], 
                      ["Derrick", "Rose"] ]

  var names = split_names.map(function(elem){
    return elem[0] + " " + elem[1];
  });

  names #=> ["Carmelo Anthony", "Rajon Rondo", "Tim Duncan", "Derrick Rose"]

In Ruby, instead of providing a block to the map method, we can also use a "shortcut", IF the block of code we want to run is a method that every element can respond to.
Let's say from the previous 2-dimensional array, we only want the last names instead of the full names.
We can grab the last names by calling #pop on every element.

  split_names = [ ["Carmelo", "Anthony"], 
                  ["Rajon", "Rondo"], 
                  ["Tim", "Duncan"], 
                  ["Derrick", "Rose"] ]

  names = split_names.map(&:pop) # shortcut for map{ |elem| elem.pop }

  names #=> ["Anthony", "Rondo", "Duncan", "Rose"]

Use modules as mixins in Ruby

Tags: ruby
Publish Date: 2014-11-04
A little revision on mixins
In many programmaing languages (such as Java), it is a common practice to use inheritance to share methods among classes.
In Ruby, however, there is another way to share methods: mixins. Let's look at the following example:
  class Animal
    def run
	  "I'm running"
	end
  end

  class Dog < Animal
  end

  class Mouse < Animal
  end

  gromit = Dog.new
	
  gromit.run
  #=> "I'm running"
All animals can run, but not all animals can hunt. So maybe it wouldn't make sense to define methods that are related to hunting in the Animal class. Instead, it might be a better idea to define these methods in a separate module.
module Hunting
  def chase_rabbit
    "I'm chasing rabbit"
  end
end
We can include module Hunting in our Dog class, so that the chase_rabbit method will be available as a instance variable. It is also worth to notice that the included module will be inserted in the class hierarchy chain.
class Dog < Animal
  include Hunting
end

gromit.chase_rabbit
#=> "I'm chasing rabbit"
Dog.ancestors.inspect
#=> [Dog, Hunting, Animal, Object, Kernel, BasicObject]
In the following example, we can see that the superclass has acquired methods by including a module. These methods can still be overriden by its subclasses. Again, please pay notice to what happens the class hierachy once a module has been included.
module Breedable
  def breed
    "breeding [original implementation]"
  end
end

class Animal
  include Breedable

  def run
	"I'm running"
  end
end

class Dog
  include Hunting

  def breed
    "breeding puppies"
  end
end

gromit = Dog.new
mickey = Mouse.new
gromit.breed #=> "breeding puppies"
mickey.breed #=> "breeding [original implementation]"

Dog.ancestors.inspect #=> [Dog, Hunting, Animal, Breedable, Object, Kernel, BasicObject]
In the previous example, we have made methods in modules available as instance methods.
However, we can also make them available as "class methods".
    module Countable
      def count_breed
        "Counting #{self.name} class"
      end
    end
  
The methods within Countable can then be included as class methods by one of the two following ways:
  class Dog
    class << self
      include Countable
    end

    include Hunting

    def breed
      "breeding puppies"
    end
  end
  Dog.count_breed #=> "Counting Dog class"
  
Or:
    class Dog
      extend Countable
      
	  include Hunting

      def breed
        "breeding puppies"
      end
    end
    Dog.count_breed #=> "Counting Dog class"
  
Again, pay attention to the class hierarchy. In this case, extend a module (or include within class << self) will not insert the module Countable in our class hierarchy.
  	Dog.ancestors.inspect #=> [Dog, Hunting, Animal, Breedable, Object, Kernel, BasicObject]