Wednesday, November 30, 2011

Perl Power - Sequential Iteration in shell scripts

Lets say you want to process a bunch of files containing a numeric sequence like "inventory000297.txt" through "inventory000302.txt". You would like to print the following:

inventory000297.txt
inventory000298.txt
inventory000299.txt
inventory000300.txt
inventory000301.txt
inventory000302.txt

I did not see a simple way to do this using a plain shell script, but adding some perl is a different story...

for F in $(perl -e 'foreach $num (297 .. 302) {printf("%s%06i%s\n","inventory",$num,".txt");}')
do
    echo ${F}
done

Notes to keep in mind:
  • $( ) contains the perl one-liner that generates the output to print
  • The perl script is between the single quotes
  • The foreach loop block is within braces { }
  • The numerical range is in parenthesis ( 297 .. 302 ), change this as necessary
  • The printf has 3 format specifiers:

    %s corresponds with "inventory", change prefix as necessary

    %06i corresponds with $num, a fixed width 6 digit integer, change as necessary

    %s corresponds with the ".txt", change extension as necessary






Adjust the size of your sequence number - if 15 digits is needed you would use %015i for the format. That leading 0 says fill in with leading zeros on output to match the specified width.

Enjoy!

Monday, August 1, 2011

Perl One-liners - Deal with the "too many files" issue

In UNIX, sometimes you end up getting a "too many files" error when using the "cp, mv, or rm" commands if there are thousands of files in a given directory. The usual workaround is to utilize the "find" command.

Without the newer (GNU) find versions, like I don't have... I have to resort to using this type of syntax to display files only in the current directory (I don't want it to descend in to sub-directories). Let say there are tons of "xml" files in the current directory.

#Example using find that will count the number of xml files
find . \( ! -name . -prune -name "*.xml" \) -type f -print | wc -l

Here is an alternative way to count the files in Perl:

perl -e '@a=<*.xml>;map {print "$_\n";} @a;' | wc -l
#Even quicker version
perl -e '@a=<*.xml>;printf("%s\n", scalar(@a));'

The caveat for the Perl version is that you would need enough memory to hold the names of the files in the array @a .

Display all Standard Perl Modules available

Here is a quick way to see all names of the Standard Perl Modules that are installed on your system:

perl -MFile::Find=find -MFile::Spec::Functions -Tlw -e 'find { wanted => sub { print canonpath $_ if /\.pm\z/ }, no_chdir => 1 }, @INC'

Thursday, April 7, 2011

Korn Shell Perl Putty Pass-thru Source Code Utility

Teleport Source Code from the UNIX world to the Windows world.

Let say you work on a UNIX system via a PC using the "Putty" terminal emulator.
You don't have a good way to print on UNIX, so you typically have to first transfer your source code to the PC so that you can print it.

Putty allows a printer pass-thu feature. By sending the proper escape sequences you can turn on printer pass-thru mode, send text to your PC, then send the escape sequences to turn off that mode. I have put this in a script form with 3 different ways of running it.
Pre-requisites:
Setup a "Generic / Text Only" printer on your PC that will print to a file.
Configure Putty to use this Generic printer for printer pass-thru.


#!/bin/ksh 
#Script:  toputty.ksh
#Purpose:  This script allows printing to a "Generic / Text Only" printer from putty 3 ways:
# 1) Piping the output from a command:               cmd | toputty.ksh"
# 2) Piping a list of filenames to print:    listoffiles | toputty.ksh -f"
# 3) Passing a file specification to print:  toputty.ksh -f "*.txt"     
#
# Note: option 1 outputs exact text from the command
# Note: options 2 and 3 include a full path filename and date time header
#
# Assumptions: filenames are based on the current directory.

#Function to construct a full pathname for the given filename $i
fullpath ()
{
s=$i                                    #s is a string containing the filename
ts=`echo $s  | awk '{print substr($0,1,1)}'`    #ts is the 1st char of string
if [[ "$ts" = "/" ]]                    #Is leading char a slash
then
  #No change is necessary because it is already a full path
  echo $s
else
  #Tack on current directory without the leading ./
  ts=`echo $s  | awk '{print substr($0,1,2)}'`          #1st two char of string
  if [[ "$ts" = "./" ]]                 # does it start with "./"
  then
    # Replace the dot with the current working directory and tack on
    # the string from the second character to the end of the string
    s=$(echo `pwd``echo $s | awk '{printf ("%s", substr($0,2))}'`)
    echo $s
  else
    #Just a filename was given so build a string from the current working
    #directory, a slash and then the filename
    s=$(echo `pwd`/`echo $s | awk '{printf ("%s", substr($0,1))}'`)
    #echo $s
  fi
fi
}

USAGE="no parameter USAGE:  command_stdout | toputty.ksh\n 1 parameter  USAGE:  ListOfFiles_stdout | toputty.ksh -f\n 2 parameter  USAGE:  toputty.ksh -f \"filespec\" \n"

TP_DIR=/tmp
TMPPUTTY=${TP_DIR}/toputty.$$

##
## Command output piped through | toputty.ksh
##
if [[ $# = "0" ]]; then                 #using STDIN for file content
   #Example:  cal | toputty.ksh
   cat - > $TMPPUTTY

##
## List of files piped through | toputty.ksh -f
##
elif [[ $# = "1" ]]; then
   if [[ $1 = "-f" ]]; then             #STDIN is a list of filenames
       #find . -type f -name "*.ksh" -print | toputty.pl
       #ls -1 *.ext | toputty.pl
       cp /dev/null $TMPPUTTY

       #For all filenames given from standard input
       for i in `cat -`
       do
          fullpath             #Ensure that each file includes its full pathname
          echo "- - - - - - - - - - - - - - - - - - - - - - - - - - - -" >> $TMPPUTTY
          echo "File: $s\t\t`date "+(date: %m/%d/%Y %H:%M:%S)"`" >> $TMPPUTTY
          echo "- - - - - - - - - - - - - - - - - - - - - - - - - - - -" >> $TMPPUTTY
          cat $s >> $TMPPUTTY
          echo "" >> $TMPPUTTY
       done
   else
       echo $USAGE
       exit 1
   fi

##
## toputty.ksh -f "filespecification"
##
elif [[ $# = "2" ]]; then       
   if [[ $1 = "-f" ]]; then             #$2 param is a filename or wildcard in double quotes
       cp /dev/null $TMPPUTTY

       #For all filenames given from standard input
       for i in `ls -1 $2`
       do
          fullpath             #Ensure that each file includes its full pathname
          echo "- - - - - - - - - - - - - - - - - - - - - - - - - - - -" >> $TMPPUTTY
          echo "File: $s\t\t`date "+(date: %m/%d/%Y %H:%M:%S)"`" >> $TMPPUTTY
          echo "- - - - - - - - - - - - - - - - - - - - - - - - - - - -" >> $TMPPUTTY
          echo $i
          cat $s >> $TMPPUTTY
          echo "" >> $TMPPUTTY
       done
   else
       echo $USAGE
       exit 1
   fi
else
   echo $USAGE
   exit 1
fi

cat $TMPPUTTY | /usr/bin/perl -e '$pc_on = "\033[5i"; $pc_off = "\033[4i"; @input = <>; $lines = @input; print "pass-through printing on\n"; print "$pc_on"; print @input; print "$pc_off"; print "pass-through printing off\n"; print $lines, " lines printed\n";'

#Clean up temp file
rm  $TMPPUTTY

Here are some possible senarios:

==> cat filename1.txt | toputty.ksh
==> ls -ltr | toputty.ksh
==> cal 2011 | toputty.ksh

Let's say you have a list of files in a text file that are in the current directory:
==> cat mylist.txt
payroll_report.pl
gl_report.ksh
ar_report.awk
hr_report.sql

==> cat mylist.txt | toputty.ksh -f

Let's say you want to print all files in the current directory with a .ksh extension:

==> toputty.ksh -f "*.ksh"

After running "toputty.ksh" similar to the 3 options, one text file will "pass-thru" the Putty terminal emulator to your PC. A DOS window will pop up, prompting you to enter a filename to save the text file as.

The file can then be printed from your PC.

An optional step you could also try is to check out a very useful "Print Spooler" utility called PrintFile. Here is the link:   PrintFile. Easy file printing in Windows - Peter Lerup
PrintFile is a super utility!!!

Create a directory on your PC called:   "C:\spooler"
Setup printfile to monitor that "C:\spooler" directory as your spool directory
Then when you use "toputty.ksh" save the file in the pop-up window as:  "C:\spooler\myfile.txt"
This will automatically print to your HP LaserJet printer in the format you have setup printfile.

Good luck and Enjoy!