Ascend Archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
(ASCEND) MAX log parsing
This might help you.
Claudio Stifel
Intercity
#!/usr/local/bin/perl
# $Id: radacct,v 1.20 1996/12/06 20:12:00 cshenton Exp $
#
# Compile usage statistics from RADIUS detail files
# and produce three different reports. See usage/help section.
#
# 1996.10.07 Chris Shenton <Chris@i3inc.com>
# Wrote radreduce, to do initial data reduction for each event.
# 1996.10.21 Chris Shenton <Chris@i3inc.com>
# Wrote session collation and grand total reports.
# 1996.11.15 Chris Shenton <Chris.Shenton@hq.nasa.gov>
# Added Ascend Max parsing, HQ X.500 lookups and Code reports.
#
# TO DO:
# - Show CLID, Session type.
# - Show usage by day-of-week, possibly by hour within day of week.
# - Genericize my stored attributes to make them Max/PortMaster-neutral.
# - Parameter-ize operation to make it more robust
# - Add hooks to handle Ascend detail files; that's why we need parametrics!
###############################################################################
# Here's what I want to capture:
# 1. Username
# 2. Date-start
# 3. Time-start
# 4. Date-stop
# 5. Time-stop
# 6. Time-used
# 7. Bytes-in (KB) Ascend
# 8. Bytes-out (KB) Ascend
# 9. Packets-in (K?) Ascend
# 10. Packets-out (K?) Ascend
# 11. CLID Ascend
# 12. IPaddress (for security)
# 13. Session type (telnet, PPP, etc)
# 14. Code (for Code Report) HQ-only
###############################################################################
require "getopts.pl";
$Detail = "detail"; # Should get this from command line too??
$ATTR_SEP = "\n"; # Delimit each attr/value pair
$KB = 1024; # Bytes/KByte;
$US_SEP = "\f"; # Formfeed between each user's sessions
$DelimRec = "\0"; # Delimit each users' events
$/ = ''; # Read paragraph-at-a-time
%MonthNum = (
'Jan', 1, 'Feb', 2, 'Mar', 3, 'Apr', 4, 'May', 5, 'Jun', 6,
'Jul', 7, 'Aug', 8, 'Sep', 9, 'Oct', 10, 'Nov', 11, 'Dec', 12 );
&Getopts("cC:dD:eE:pP:sS:tT:h");
if ($opt_h) {
die "
Compile usage statistics from RADIUS detail file(s) and generate reports.
Parse the detail file's multi-line data per start/stop event.
Combine related start-stop data into one record, then aggregate
and collate events into sessions, cluster sessions by users,
calculating running totals, and finally grand totals.
Usage: radacct [-e|-Eefile] [-d|Ddfile] [-p|Ppfile] [-s|-Ssfile] [-t|-Ttfile] detailfile ...
You can have multiple reports generated simultaneously.
Capitalized options take filename arguments for their output data.
-e|-E: events (starts and stops) sorted as the log file is.
This is the simplest data-reduction,
suitable for further postprocessing, eg: for port-utilization.
-s|-S: sessions (start to stop) sorted by user (separated by form-feed).
Each user cluster is a log of usage, like on a phone bill;
and total times are cumulative leading to a total on the last.
-t|-T: total time per user (minutes) with number of sessions and average time.
Most concise report, suitable for import into billing software.
-d|-D: daily usage by hour.
-p|-P: port usage over all ports.
-c|-C: HQ Code usage; can take a while for X.500 lookups.
";
}
unless ($opt_c || $opt_C ||
$opt_d || $opt_D ||
$opt_e || $opt_E ||
$opt_p || $opt_P ||
$opt_s || $opt_S ||
$opt_t || $opt_T) {
die "
Usage: radacct [-e|-Eefile] [-h|-Hhfile] [-p|-Ppfile] [-s|-Ssfile] [-t|-Ttfile] detailfile ...
radacct -h (for detailed help)
";
}
# Open any output files to make sure we can write.
if ($opt_C) {
$opt_c = 1;
}
else {
$opt_C = "&STDOUT";
}
open(CFILE, ">$opt_C")
|| die "ERROR: could not open cfile \"$opt_C\"";
if ($opt_D) {
$opt_d = 1;
}
else {
$opt_D = "&STDOUT";
}
open(DFILE, ">$opt_D")
|| die "ERROR: could not open dfile \"$opt_D\"";
if ($opt_E) {
$opt_e = 1;
}
else {
$opt_E = "&STDOUT";
}
open(EFILE, ">$opt_E")
|| die "ERROR: could not open efile \"$opt_E\"";
if ($opt_P) {
$opt_p = 1;
}
else {
$opt_P = "&STDOUT";
}
open(PFILE, ">$opt_P")
|| die "ERROR: could not open pfile \"$opt_P\"";
if ($opt_S) {
$opt_s = 1;
}
else {
$opt_S = "&STDOUT";
}
open(SFILE, ">$opt_S")
|| die "ERROR: could not open sfile \"$opt_S\"";
if ($opt_T) {
$opt_t = 1;
}
else {
$opt_T = "&STDOUT";
}
open(TFILE, ">$opt_T")
|| die "ERROR: could not open efile \"$opt_T\"";
while ($ARGV[0]) {
$Detail = shift;
open(DETAIL, $Detail)
|| die "Could not open detail file \"$Detail\"\n";
printf STDERR "INFO: working on detail file \"$Detail\"...\n";
# From each record, gather attributes and associate with the user-session.
# The accumulated data is stored in an Assoc indexed by user-session;
# the data itself is a string-representation of the attribute/value
# pair assoc. Sheesh.
while (<DETAIL>) {
chop;
# Place fields into handy generic variables.
($Ddd, $Mmm, $Dd, $Time, $Yyyy) =
/^(\w+)\s+(\w+)\s+(\d+)\s+([\d:]+)\s+(\d+)/;
$Date = sprintf("%4d/%2.2d/%2.2d",
$Yyyy, $MonthNum{$Mmm}, $Dd);
($Acct_Session_Id) = /Acct-Session-Id = \"([0-9A-F]+)\"/;
($Acct_Session_Time) = /Acct-Session-Time = (\d+)/; # Stop only
($Acct_Status_Type) = /Acct-Status-Type = (\w+)/; # Start/Stop
($Client_Port_Id) = /Client-Port-Id = (\d+)/;
($Framed_Address) = /Framed-Address = ([\d.]+)/;
($User_Name) = /User-Name = \"([^"]+)\"/; # should be better??
# Ascend only.
($Client_Port_Id) = /NAS-Port = (\d+)/; # ??different attr name than Livingston :-(
($InOctets) = /Acct-Input-Octets = (\d+)/;
($OutOctets) = /Acct-Output-Octets = (\d+)/;
($InPackets) = /Acct-Input-Packets = (\d+)/;
($OutPackets) = /Acct-Output-Packets = (\d+)/;
($Caller_Id) = /Caller-Id = "([^"]*)"/;
if ($opt_e) {
write EFILE;
}
# Make a user-session key from the username and session id.
$USkey = "$User_Name:$Acct_Session_Id";
# Find any attributes we may already have and make into an assoc.
$Attrs = $USattr{$USkey}; # Probably empty if this is the START record
%Attr = &radacct_attr_make_assoc($Attrs);
# Add attributes found in this record.
if ($Acct_Status_Type eq 'Start') {
if ($Attrs) {
printf STDERR "WARN: Start record found but attrs already instantiated\n$_\n";
}
$Attr{'start-date'} = $Date;
$Attr{'start-time'} = $Time;
$Attr{'Acct-Session-Id'} = $Acct_Session_Id;
$Attr{'Client-Port-Id'} = $Client_Port_Id;
$Attr{'Framed-Address'} = $Framed_Address;
$Attr{'User-Name'} = $User_Name;
$Attr{'Caller-Id'} = $Caller_Id;
}
elsif ($Acct_Status_Type eq 'Stop') {
if (! $Attrs) {
#??printf STDERR "WARN: Stop record found but attrs not instantiated\n$_\n";
$Attr{'Acct-Session-Id'} = $Acct_Session_Id;
$Attr{'Client-Port-Id'} = $Client_Port_Id;
$Attr{'Framed-Address'} = $Framed_Address;
$Attr{'User-Name'} = $User_Name;
}
$Attr{'stop-date'} = $Date;
$Attr{'stop-time'} = $Time;
$Attr{'Acct-Session-Time'} = $Acct_Session_Time;
# Ascend only.
$Attr{'Acct-Input-Octets'} = $InOctets;
$Attr{'Acct-Output-Octets'} = $OutOctets;
$Attr{'Acct-Input-Packets'} = $InPackets;
$Attr{'Acct-Output-Packets'} = $OutPackets;
}
else {
printf STDERR "Bogus start/stop: $_\n";
}
# Convert the %Attr to string so we can update the user-session assoc.
$USattr{$USkey} = &radacct_attr_make_string(%Attr);
}
}
# List each users-session sorted by username
# and caculate running totals for connection time.
foreach $USkey (sort keys %USattr) {
%Attrs = &radacct_attr_make_assoc($USattr{$USkey});
# ?? Need to handle session which Start but do not Stop
# If this session user is different than last, save/print old info.
$User = $Attrs{'User-Name'};
if ($User ne $UserLast) { # First session for this user
$Sessions{$User} = 1;
$TimeTotal{$User} = $Attrs{'Acct-Session-Time'};
$InOctetsTotal{$User} = $Attrs{'Acct-Input-Octets'}; # Ascend
$OutOctetsTotal{$User} = $Attrs{'Acct-Output-Octets'}; # Ascend
$InPacketsTotal{$User} = $Attrs{'Acct-Input-Packets'}; # Ascend
$OutPacketsTotal{$User} = $Attrs{'Acct-Output-Packets'}; # Ascend
$UserLast = $User;
if ($opt_s) {
printf(SFILE "%s\nUserName Start_Date_and_Time Stop_Date_and_Time_ Secnds TotSecnd IP-Address_____ Pt___ Sess-ID__ KB_In_ KB_Out PktIn_ PktOut Cal
lerID__\n", $US_SEP);
}
}
else { # Subsequent user session; accumulate
$TimeTotal{$User} += $Attrs{'Acct-Session-Time'};
$Sessions{$User}++;
$InOctetsTotal{$User} += $Attrs{'Acct-Input-Octets'}; # Ascend
$OutOctetsTotal{$User} += $Attrs{'Acct-Output-Octets'}; # Ascend
$InPacketsTotal{$User} += $Attrs{'Acct-Input-Packets'}; # Ascend
$OutPacketsTotal{$User} += $Attrs{'Acct-Output-Packets'}; # Ascend
}
if ($opt_d) { # Compile port-usage info
($HourStart) =
($Attrs{'start-time'} =~ /^(\d+):\d+:\d+/);
($HourStop) =
($Attrs{'stop-time'} =~ /^(\d+):\d+:\d+/);
#print "P: START=$HourStart, STOP=$HourStop\n";
if ($HourStart || $HourStop) { # Cope with missing start or stop
if (! $HourStop) {
$HourStop = $HourStart;
}
elsif (! $HourStart) {
$HourStart = $HourStop;
}
for ($Hour = $HourStart; $Hour <= $HourStop; $Hour++) {
$Hourz = sprintf("%02d", $Hour); # Ensure leading zero
$Hours{$Hourz}++;
#print "P: Hour=$Hourz, Hours=$Hours{$Hourz}\n";
}
}
}
if ($opt_p) { # Ports usage
$Ports{$Attrs{'Client-Port-Id'}}++;
}
if ($opt_s) {
printf(SFILE
"%-8s %10s %8s %10s %8s %6d %8d %-15s %5d %9s %6d %6d %6d %6d %s\n",
$Attrs{'User-Name'},
$Attrs{'start-date'},
$Attrs{'start-time'},
$Attrs{'stop-date'},
$Attrs{'stop-time'},
$Attrs{'Acct-Session-Time'}, # seconds!
$TimeTotal{$User},
$Attrs{'Framed-Address'},
$Attrs{'Client-Port-Id'},
$Attrs{'Acct-Session-Id'}
,
$Attrs{'Acct-Input-Octets'}/$KB, # Ascend
$Attrs{'Acct-Output-Octets'}/$KB, # Ascend
$Attrs{'Acct-Input-Packets'}, # Ascend
$Attrs{'Acct-Output-Packets'}, # Ascend
$Attrs{'Caller-Id'} # Ascend
);
}
}
# Display daily usage by hour.
if ($opt_d) {
# First find maximum/totals to determine scaling
foreach $Hour (keys %Hours) {
$HoursTotal += $Hours{$Hour};
if ($Hours{$Hour} > $HoursMax) {
$HoursMax = $Hours{$Hour};
}
}
printf(DFILE "Hr Sessions %%Sess %%Graph\n");
foreach $Hour (sort keys %Hours) {
$HoursMaxPercent = 100 * $Hours{$Hour} / $HoursMax;
$HoursTotPercent = 100 * $Hours{$Hour} / $HoursTotal;
printf(DFILE "%2s %8d %5.1f %-s\n",
$Hour, $Hours{$Hour},
$HoursTotPercent, "=" x ($HoursMaxPercent / 5));
}
printf(DFILE "Total session hours: $HoursTotal\n");
}
# Display port usage.
if ($opt_p) {
$Types{"1"} = "Digital";
$Types{"2"} = "Analog";
printf(PFILE "Port_ Type___ Line Channel Number\n");
foreach $Port (sort keys %Ports) {
if (($Type, $Line, $Channel) = ($Port =~ /^(\d)(\d\d)(\d\d)/)) {
printf(PFILE "%5s %-7s %4s %7s %5d\n",
$Port, $Types{$Type}, $Line, $Channel, $Ports{$Port});
$PortsTotal += $Ports{$Port};
}
}
printf(PFILE "Total Port sessions: %6d\n", $PortsTotal);
}
# Display time total (minutes) for each user.
if ($opt_t) {
printf(TFILE "UserName MinsTotal Sessions MinsAvg __KBin _KBout _PktIn PktOut\n");
foreach $UserKey (sort keys %TimeTotal) {
printf(TFILE "%-8s %9.1f %8d %7.1f %6d %6d %6d %6d\n",
$UserKey,
$TimeTotal{$UserKey}/60,
$Sessions{$UserKey},
$TimeTotal{$UserKey}/60/$Sessions{$UserKey}
,
$InOctetsTotal{$UserKey}/$KB,
$OutOctetsTotal{$UserKey}/$KB,
$InPacketsTotal{$UserKey},
$OutPacketsTotal{$UserKey},
);
}
}
# NASA/HQ: Display users by department Code with Code totals.
if ($opt_c) {
foreach $UserKey (sort keys %TimeTotal) {
$Username = $UserKey; # merge this in later.
$Time = $TimeTotal{$UserKey}/60;
if ($Username && $Time) {
$UserTime{$Username} = $Time;
$Code = &x500_get_code($Username);
if (! $CodeUsers{$Code}) {
$CodeUsers{$Code} = $Username;
}
else {
$CodeUsers{$Code} .= " $Username";
}
$CodeTime{$Code} += $Time;
}
else {
print STDERR "WARN: No username=\"$Username\" or time=\"$Time\" found; ignored.\n";
}
}
foreach $Code (sort keys %CodeUsers) {
@Users = split(/\s+/, $CodeUsers{$Code});
foreach $User (@Users) {
printf(CFILE "%-24s %-10s %7.1f\n", $Code, $User, $UserTime{$User});
}
printf(CFILE "------------------------ ---------- -------\n");
printf(CFILE "%-24s %-10s %7.1f\n\n", $Code, "Total", $CodeTime{$Code});
}
}
###############################################################################
# Make an Assoc into a simple string
# suitable for assigning to an Assoc or DBM value.
sub radacct_attr_make_string {
local(%Assoc) = @_; # Assoc of Attr/Value pairs
return join($ATTR_SEP, %Assoc);
}
###############################################################################
# Make a string of into assoc.
sub radacct_attr_make_assoc {
local($Assocs) = @_; # Assoc of Attr/Value pairs
return split($ATTR_SEP, $Assocs);
}
###############################################################################
# Search X.500 by username and return Code.
sub x500_get_code {
local($Username) = @_;
local($X500s, $Entries, @Entries, $Entry, $Org, $Emp);
$X500s = `finger $Username\@x500.hq.nasa.gov`; # Filter this!!??
$Entries = @Entries = split(/\n\n/, $X500s);
foreach $Entry (@Entries) {
if ($Entry =~ /\s$Username\n/) {
($Org, $Emp) = ($Entry =~ /Organization: (.+), Employer: (.*)\n/);
if ($Emp eq "NASA") {
return "Code $Org";
}
else {
return $Emp;
}
}
}
return "UNKNOWN";
}
###############################################################################
# Event report.
format EFILE_TOP =
Date Time Sess-Id User-Name Stat Pt IP-Address TimeOn