Logging and Make

• References
• Logging
  • What should servers log?
  • How should servers log?
  • Appending to a log file
  • System logging facility
  • syslog priorities
  • Facilities
  • Sample log entries
  • Types of logs
  • Log file analysis
• Make
  • Large projects with Make
  • Make extensions
  • Using sh with Make

References:

Man pages for syslog(3), syslog.conf(5), syslogd(8)

Logging

Why should servers log?

• Performance tuning
• Upgrade justification
• Problem tracking

What should servers log?

Depends on the service.

Normally a log entry contains several pieces of information:

• Date and time
• Service that caused the entry
• Client address that caused the entry
• Host on which the server runs
• Event

How should servers log?

Basic choices:

• Appending to a log file
• System logging facility

Appending to a log file

Creating a log entry: (Simple version)

1. Server opens log file for append
2. Server writes log entry
3. Server closes log file

Problems with this?

• Concurreny: Two server processes writing to the same logfile
• Performance: Every log entry requires an open() and close() call

Some solutions to these problems:

• Rely on Unix to perform atomic appends to the log file. (One write() per log entry)
• Do all logging in the parent. (keep log open)

System logging facility

Unix provides a flexible logging facility:

syslog

The following calls are used:

• openlog(char *identifier, int logoptions, int facility)
• syslog(int priority, char *message, ...)
• closelog()
• setlogmask(int maskpriority)

syslog uses a Unix domain socket to talk to the logging daemon: syslogd

It solves both the concurrency and performance problems.

syslog priorities

Priorities are encoded as a facility and a level.

Levels are:

• LOG_EMERG Panic conditions
• LOG_ALERT Urgent things
• LOG_CRIT Critical stuff
• LOG_ERR Errors
• LOG_WARNING Warnings
• LOG_NOTICE None errors
• LOG_INFO Information
• LOG_DEBUG Debugging

Log options:

• LOG_PID Add pid to log entry
• LOG_CONS Write to console
• LOG_NDELAY Open immediately

Facilities

• LOG_KERN Kernel messages
• LOG_USER Random user processes
• LOG_MAIL The mail system
• LOG_DAEMON System daemons
• LOG_AUTH Login, su, getty
• LOG_LPR Printing system
• LOG_NEWS USENET news
• LOG_UUCP Never used
• LOG_CRON Cron system
• LOG_LOCAL0-7 Reserved for local use

Typical usage:

openlog("myservice", LOG_PID, LOG_DAEMON);

syslog(LOG_ERR, "Sorry, error #%d", errno);

Sample log entries

Generated by syslogd:

Mar 12 08:20:06 sciences pppd[28492]: ioctl(SIOCSIFASYNCMAP): No such device or address

Generated by httpd:

rohan.sdsu.edu - - [17/Feb/1995:00:04:47 -0800] "GET / HTTP/1.0" 200 4945

Types of logs

Two basic types:

• Plain text
• Binary (record based)

Plain text is preferable, but there are cases where record based logs are desirable:

• Reducing size of log entries
• Generating log entries at a rapid rate
• Reduce overhead

To analyze these logs, you need tools.

Examples:

• POP server
• Audit trail
• System Accounting

Log file analysis

When logs are used as a justification tool, reports need to be generated.

SDSUanalyze:

Generate a graph of average usage per time interval:

Report of usage for the last 5 days in intervals of 60 minutes

Time Logins Relative logins

00:00 (150) *******

01:00 (146) *******

02:00 (147) *******

03:00 (137) *******

04:00 (132) ******

05:00 (122) ******

06:00 (156) ********

07:00 (210) ***********

08:00 (445) ***********************

09:00 (562) *****************************

10:00 (688) ************************************

11:00 (769) ****************************************

12:00 (794) *****************************************

13:00 (1139) ************************************************************

14:00 (994) ****************************************************

15:00 (744) ***************************************

16:00 (475) *************************

17:00 (257) *************

18:00 (210) ***********

19:00 (177) *********

20:00 (166) ********

21:00 (164) ********

22:00 (155) ********

23:00 (148) *******

9034 succesful logins, 53 unsuccesful logins

Make

A tool to build projects.

Typical Makefile:

#

# Makefile for project X

# 4/1/1942

#

CFLAGS= -g -I/usr/local/include

LIBS= -lresolv

CC= gcc

OBJS= file1.o file2.o

TARGET= X

$(TARGET): $(OBJS)

$(CC) -o $@ $(OBJS)

Large projects with Make

Large projects have:

• multiple directories
• libraries

Each directory needs to have its own Makefile, even documentation directories.

Each Makefile needs a standard set of targets to include:

• all
• install
• clean
• depend

Make extensions

Each Makefile can include a configuration file:

#

# Makefile for program Y in project X

# 4/1/42

#

include ../Makefile.config

TARGET= Y

OBJS= Y1.o Y2.o

all: $(TARGET)

$(TARGET): $(OBJS)

$(CC) -o $@ $(OBJS)

#

# Makefile.config for project X

# 4/1/42

#

CC= gcc

CFLAGS= -g

Using sh with Make

In the project's main Makefile, you can use something like the following:

#

# Main Makefile for project X

# 4/1/42

#

DIRS= src lib doc db

all:

@for dir in $(DIRS); \

(cd $$dir; $(MAKE) $(MFLAGS) all); \

done;

install:

@for dir in $(DIRS); \

(cd $$dir; $(MAKE) $(MFLAGS) install); \

done;

etc...