* Fix for memory leaks in trusted ca load functionality.
* Integrated latest webfiltering engine.
SafeSquid produces logs in three distinct formats.
The access.log has been traditional favorite, because it can be used by a variety of log analyzers like Calamaris, SARG, Squint, SquidTailD, etc.
The reports produced by these log analyzers reveal useful details of the overall usage and the pattern of access of the application.
See Sample Log Reports
Access Log fields:
start_time_in_seconds.milliseconds elapsed_time client cachecode/status size method url username peercode/peer mime
1189403858.675 654 192.168.0.21 TCP_MISS/200 246 GET http://ds.ds3ps.co.uk:80/refer/surebrowse/operator/chat-server.xml?time=1189404101675 sudipta DIRECT/ds.ds3ps.co.uk text/xml
The details of the fields in access.log are as follows:
|Time||UNIX time stamp as Coordinated Universal Time (UTC) seconds with a millisecond resolution.|
|Elapsed Time||Length of time in milliseconds that the cache was busy with the transaction. The information is logged after the reply has been sent, not during the lifetime of the transaction.|
|Client||IP address of the requesting host.|
|Cachecode/Status||Two entries separated by a slash. Code specifies the result of the transaction: the kind of request, how it was satisfied, or in what way it failed. The second entry contains the HTTP result codes.|
|Bytes||Amount of data delivered to the client. This does not constitute the net object size, because headers are also counted. Also, failed requests may deliver an error page, the size of which is also logged here.|
|Method||Request method to obtain an object, e.g. GET, POST, CONNECT.|
|Peerstatus/Peerhost||Two entries separated by a slash. The first entry represents a code that explains how the request was handled, for example, by forwarding it to a peer, or returning the request to the source. The second entry contains the name of the host from which the object was requested. This host may be the origin site, a parent, or any other peer. Also note that the host name may be numerical.|
|Mime||Mime type of the object.|
The extended.log (NCSA / Extended log format) records maximum details of each request handled by the proxy application.
Log Analyzers like Sawmill can generate analysis reports using the extended log, and give lots more information, than the ones using access.log.
See Sawmill sample report.
"UNIQUE_RECORDID" ELAPSED_TIME_IN_MSEC CLIENT_IP "USER_NAME" "CLIENT_CONNECTION_ID" [DATE_TIME_OF_REQUEST] "METHOD URL" "HTTP_STATUS_CODE" BYTES_TRANSFERRED "REFERRER_URL" "USER_AGENT" MIME_TYPE "FILTER_NAME FILTERING_REASON" "COMMA_SEPARATED_LIST_OF_PROFILES_APPLIED" "INTERFACE_IP:INTERFACE_PORT"
"1191586598.504-7-192.168.0.221-8080" 929 192.168.0.150 "anonymous" "7" [05/Oct/2007:17:46:39] "GET http://updates.f-prot.com:80/cgi-bin/check-updates?run_as=check_updates&protocol=1" 200 750 "-" "FPAV_Update_Monitor/3.16f (Windows; WINNT; 2000 Professional; SP4)" text/plain "- -" "-" "192.168.0.221:8080"
The details of the fields in extended.log are as follows:
|Unique Record ID||A unique record identifier, to prevent duplication of records when imported into SQL databases.Here in e.g. 1215419711.460|
|Elapsed time in milliseconds||Elapsed time of the request, in milliseconds.|
|Client IP||The IP address of the requesting client.|
|User name||The username, (or user ID) used by the client for authentication. If no value is present, "anonymous" is substituted.|
|Client connection ID||The internal SafeSquid ID associated with this connection.|
|Date & time of request||
The date and time stamp of the HTTP request.The fields in the date/time field are [dd/MMM/yyyy:hh:mm:ss +-hhmm], where the fields are defined as follows:
dd is the day of the month, MMM is the month, yyyy is the year, hh is the hour, mm is the minute, ss is the seconds.
|Method URL||The HTTP request. The request field contains three pieces of information. The main piece is the requested resource. The request field also contains the HTTP method.|
|HTTP Status Code||The status code is the numeric code indicating the success or failure of the HTTP request.|
|Bytes Transferred||This field is a numeric field containing the number of bytes of data transferred as part of the HTTP request, not including the HTTP header. E.g. 750.|
|Referrer URL||The referrer is the URL of the HTTP resource that referred the user to the resource requested. "-" is substituted when there are no referrers.|
|User agent||An HTTP client that makes HTTP requests. It is customary for an HTTP client, such as a Web browser, to identify itself by name when making an HTTP request. It is not required, but most HTTP clients do identify themselves by name.|
|Mime type||MIME-type of the requested object. E.g. text/plain.|
|Filter name & Filtering reason||If the request get blocked, then this field contains the name of the filter, or the reason for which the request was blocked. "- -" is substituted when there are no blocks.|
|Comma separated list of profiles applied||Comma separated list of profiles that were applied to the request. "-" is substituted when no profiles are applied.|
|Interface IP:Interface port||IP:PORT that received the request. This can be important when SafeSquid is listening on multiple IPs or Ports.|
This is SafeSquid's native log format.
It records various functional aspects like REQUESTS, SECURITY, REDIRECT etc. that are effected by the various features and their configuration.
You can control the verbosity of the Native log by specifying LOGLEVEL, as shown in the table below.
The LOGLEVEL parameter affects only the SafeSquid's Native log.
|Value||Process logged||Value||Process logged|
|8||Header filtering||131072||External parsers|
|32||Cookie filtering||524288||DNS blacklist|
So, if you wish to record only the requests set LOGLEVEL to 1, if you wish to record only caching related activities set LOGLEVEL to 2048.
If you wish to record all the three activities of rewriting, limits and forwarding, you would simply set LOGLEVEL to 512 + 1024 + 16384 i.e. 17920.
Similarly, if you wished to view absolutely everything (and run the risk of generating a very huge log file in a very short time!), you could set LOGLEVEL to a total of all the values in the table, i.e. 134217727 which is also the default LOGLEVEL if you simply comment the LOGLEVEL specification!.
If you wished to produce just debug logs you should set the LOGLEVEL to 134217728.
If you wished to record all activities and debug information, you should set the LOGLEVEL to 268435455.
NOTE: Adjusting this value requires a restart of SafeSquid service.
There obviously needs to be a control on log file size. SafeSquid executable cannot start if the size of any of the log files exceeds 2147483648 bytes (2GB).The parameter <LOG_SIZE_LIMIT> sets the maximum size in bytes for a log file, exceeding which, the <INIT_SCRIPT> logrotate (/etc/init.d/safesquid logrotate) will automatically truncate and compress all the three types of log files. The same command can be also run manually to rotate your logs in case any situation demands.