Examples¶
Create frequency graph from a log¶
Task: Read /var/log/dpkg.log and create a graph to visualize how often packages are installed, upgraded and removed.
Solution: The loop (34) calls function read_log which reads the log line by line (13), splits the fields (14) and concatenate date l[0] and time l[1] in minutes (15). Third field of the log l[2] is status of the dpkg operation(install, upgrade, remove …). Method zincrby (16) increments by 1 the score of word in the key l[2]. As a result the database contains keys(install, upgrade, remove …) and associated lists of words sorted by score. Next loop (38) calls the function write_csv with all keys. As a result status.csv files are created in the current directory with the (date;score) pairs.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 | #!/usr/bin/ruby
# Tested with ruby 2.3.3, ruby-redis 3.3.3 and redis 4.0.1
require 'redis'
LOG_FILES = ['/var/log/dpkg.log', ]
LOG_SEPARATOR = ' '
CSV_SEPARATOR = ';'
def read_log(log_file, r)
# This function reads log_file and put the status into the database
f = File.open(log_file, "r")
f.each do |line|
l = line.split(LOG_SEPARATOR)
word = l[0] + " " + l[1][0..-4]
r.zincrby(l[2], 1, word)
end
f.close
end
def write_csv(status, r)
# This function reads the database and writes the status CSV file
f = File.open(status + ".csv", "w")
l = r.zrange(status, 0, -1, :with_scores => true)
l.each do |x|
f.write( x[0] + CSV_SEPARATOR + x[1].to_s + "\n")
end
f.close
end
r = Redis.new(host: "localhost", port: 6379, db: 0)
r.flushdb
for log_file in LOG_FILES
read_log(log_file, r)
end
for status in r.keys
write_csv(status, r)
end
|
Result: The status.csv files can be used to create a graph with gnuplot.

List 10 most used words in a text¶
Task: Read text from a file and list 10 most frequently used words in it.
Solution: Let’s use article about Redis at wikipedia.org as a text.
#!/bin/bash
lynx -dump -nolist https://en.wikipedia.org/wiki/Redis > redis.txt
zincrby (12) increments by 1 the score of word in the key topchart and zrange (14) returns top 10 words with scores.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | #!/usr/bin/ruby
# Tested with ruby 2.3.3, ruby-redis 3.3.3 and redis 4.0.1
require 'redis'
FILE = 'redis.txt'
r = Redis.new(host: "localhost", port: 6379, db: 0)
r.flushdb
f = File.open(FILE, "r")
f.read.scan(/\w+/).each { |word| r.zincrby("topchart", 1, word)}
ranking = r.zrange("topchart", -10, -1, :with_scores => true)
for x in ranking
puts ( x[1].to_s + " " + x[0] )
end
|
Result:
> ./create-topchart.rb
11.0 Retrieved
13.0 edit
23.0 in
24.0 a
24.0 is
26.0 and
33.0 of
34.0 to
37.0 the
69.0 Redis