Web Hosting Talk







View Full Version : Seeking perl solutions to pull IP out of string


dynamicnet
09-16-2009, 04:57 PM
Greetings:

Given data like the following --

09/14/2009 04:47:04.448 - Info - Network Access - ICMP packet allowed - [SRC IP], 3, X1 - [DEST IP], 3, X1 - ICMP Destination Unreachable, Code: 3
09/16/2009 15:47:04.784 - Notice - Network Access - Web access request dropped - [SRC IP], 3473, X1, [FULLY QUALIFED MACHINE NAME] - [DEST IP], 443, X1 - TCP HTTPS 14 (WAN->WAN)

-- what are the different ways using perl to pull the source IP?

Right now, I'm using


$source_ip = `echo $line_of_data | awk -F- '{print \$2}' | awk -F, '{print \$1}'`;



but when the application is run via a qmail forward file, when it hits lines with fully qualified machine names in them, the above errors with the following:

Sep 16 16:36:30 mail qmail: [QMAIL MESSAGE ID] delivery [QMAIL DELIVERY ID]: success: sh:_-c:_line_1:_syntax_error_near_unexpected_token_`('/sh:_-c:_line_1:_`echo_09/16/2009_15:47:04.784_-_Notice_-_Network_Access_-__Web_access_request_dropped_-__[SRC IP],_3473,_X1,_[FULLY QUALIFED MACHINE NAME]_-__[DEST IP],_443,_X1_-__TCP_HTTPS_14_(WAN->WAN)_|_awk_-F-_'{print_$5}'_|_awk_-F,_'{print_$1}''/Usage:_whois_[OPTION]..._OBJECT...//-l_____________________one_level_less_specific_lookup_[RPSL_only]/-L_____________________find_all_Less_specific_matches/-m_____________________find_first_level_more_specific_matches/-M_____________________find_all_More_specific_matches/-c_____________________find_the_smallest_match_containing_a_mnt-irt_attribute/-x_____________________exact_match_[RPSL_only]/-d_____________________return_DNS_reverse_d
Sep 16 16:36:30 mail qmail: [QMAIL MESSAGE ID]+elegation_objects_too_[RPSL_only]/-i_ATTR[,ATTR]...______do_an_inverse_lookup_for_specified_ATTRibutes/-T_TYPE[,TYPE]...______only_look_for_objects_of_TYPE/-K_____________________only_primary_keys_are_returned_[RPSL_only]/-r_____________________turn_off_recursive_lookups_for_contact_information/-R_____________________force_to_show_local_copy_of_the_domain_object_even/_______________________if_it_contains_referral/-a_____________________search_all_databases/-s_SOURCE[,SOURCE]...__search_the_database_from_SOURCE/-g_SOURCE:FIRST-LAST___find_updates_from_SOURCE_from_serial_FIRST_to_LAST/-t_TYPE________________request_template_for_object_of_TYPE_('all'_for_a_list)/-v_TYPE________________request_verbose_template_for_object_of_TYPE/-q_[version|sources|types]__query_specified_server_info_[RPSL_on
Sep 16 16:36:30 mail qmail: 1253133390.567373+ly]/-F_____________________fast_raw_output_(implies_-r)/-h_HOST________________connect_to_server_HOST/-p_PORT________________connect_to_PORT/-H_____________________hide_legal_disclaimers/______--verbose________explain_what_is_being_done/______--help___________display_this_help_and_exit/______--version________output_version_information_and_exit/Not_a_GLOB_reference_at_/[FULL PATH TO APPLICATION]_line_104,_<STDIN>_line_9./did_0+0+1/


Please note, the above doesn’t happen on every line of input, only specific lines where I’m guessing the use of system calls with echo, even though piped to awk, cause the problem.

When I run small tests through a test script (not going through email interception), the output of the SRC IP works 100% of the time.

What alternatives, in Perl, do I have to be able to grab the [SRC IP] given the input data streams noted above?

Thank you.

DigitalLinx
09-16-2009, 07:24 PM
You can match the IP using regex such as

$ip = $1 if $data =~ /- (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})/;
It should match [SRC IP] as it's the first match, let me know if it works or not.

dynamicnet
09-16-2009, 09:19 PM
Greetings:

Thank you for your reply.

I tested the above, and while I copied and pasted it, maybe I missed something.

When I run a test script, to try and print out the $ip I get

Use of uninitialized value

Given


$ip = $1 if $data =~ /- (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})/;


does the /- (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})/ portion need to be in single or double quotes? Or is "/" the container?

What if $1 has no value and the $data doesn't match? Is that causing the initialized value?

Thank you.

e-Sensibility
09-16-2009, 11:24 PM
You're probably getting the uninitialized value error because you don't have anything in the variable $data.

here's a complete (albeit simple) solution.


#!/usr/bin/perl
while (<>) {
if (/(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})/) {
@ip[$count++] = $1;
}
}
for (@ip) {
print "ip is $_\n";
}


so that will find one ip per line and print them all out . . . obviously this is just an example and you'll probably need to modify it to suit your needs, but hopefully it helps you get the gist.

I sent you a PM, by the way. Add me on AIM if you need some extra help . . . I should be on for a few more hours :)

DigitalLinx
09-17-2009, 11:01 AM
You're probably getting the uninitialized value error because you don't have anything in the variable $data.

here's a complete (albeit simple) solution.


#!/usr/bin/perl
while (<>) {
if (/(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})/) {
@ip[$count++] = $1;
}
}
for (@ip) {
print "ip is $_\n";
}
so that will find one ip per line and print them all out . . . obviously this is just an example and you'll probably need to modify it to suit your needs, but hopefully it helps you get the gist.

I sent you a PM, by the way. Add me on AIM if you need some extra help . . . I should be on for a few more hours :)
Yeah the above should work although it should be $ip[$count++] not @ip[$count++] and in this situation you should be using push() rather than C-style array population, but that's just perl talk (please take no offense, I'm just preaching best Perl coding practices)
Also rather than waiting for while(<>) to finish, and after loop through an array, it's much more optimal to print the $ip as data comes in, saves a lot more resources especially because mail logs can get rather big.
If you can PM me more raw uncensored data i can throw a script together for you.

e-Sensibility
09-17-2009, 01:09 PM
You're right -- but cut me some slack, it was late and I wrote the script in like 30 seconds ;)

dynamicnet
09-17-2009, 01:53 PM
Greetings:

Thank you very much for your time and responses.

I ended up using a modification of the while post with #2 post for


$ip = $1 if $line_in_body =~ (/(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})/);


Thank you!