Sunday, July 8, 2012

Shell script - Capture snippets from vi or Vim

Here is a utility that may be useful to people who use vi or Vim.
Perl Squirrel has a common tendency of collecting useful code samples and this tool enables that without too much time or effort.  The general idea is this... While editing code, highlight the code sample worth keeping and file it in a text file without leaving the vi or Vim environment. 

Here are the requirements for this utility:
  • You use 'vi' or 'Vim'
  • Create a subdirectory called "notes" under your home directory
  • Place the "snip.sh" utility in your personal 'bin' directory or somewhere in your PATH

Here is the code: (See Instructions in the code comments)
#!/bin/bash #Script: snip.sh #Purpose: Grab certain lines of text from the file # you are editing/viewing in vi/vim, # and append this snippet to a notes file. # #How to use in vi/vim session: # Position cursor on 1st line to save, press the letters # ma # Position cursor on last line to save, press the letters # mb # Capture the snippet to the default ~/notes/snippet.txt file # :'a,'b w !snip.sh # Or, Capture the snippet to a specified file ~/notes/favorite.txt # :'a,'b w !snip.sh favorite ["Optional Short description"] # #This code assumes you will store the snippets in a "notes" #subdirectory under your home directory. #Place this code in your personal bin directory, or in your PATH. #~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ SNIP=${1:-snippet} DESC="${2:-UNNAMED SNIPPET}" echo '#[SNIP]------------------------------------' >> ~/notes/${SNIP}.txt echo "#[SNIP] ${DESC} " >> ~/notes/${SNIP}.txt echo '#[SNIP]------------------------------------' >> ~/notes/${SNIP}.txt cat - >> ~/notes/${SNIP}.txt

Thursday, May 31, 2012

Perl Power - Remove files fast

Perl Squirrel has been very busy collecting nuts ( about 1.5 million ) over the past years and his store room was about to burst. If the nuts were actually files in a directory, how would Nugget get rid of them fast? Nugget wants to keep only the nuts that were collected in the past 90 days ( they would taste better ). First get a count of the files that would get removed using the Unix find command and mtime option to count files older than 90 days...

cd \perlsquirrel\storeroom find . \( ! -name . -prune "*nuts.*" \) -type f -mtime +90 -print | wc -l
Now lets say the number of files returned was 1 million. In order to remove these, Nugget did the following...
cd \perlsquirrel\storeroom find . \( ! -name . -prune "*nuts.*" \) -type f -mtime +90 -print | perl -lne unlink
This removed 1 million files in about 9 minutes.
Note: This find command prevents descending into subdiretories.
Perl Squirrel advises...measure twice...cut once.

Sunday, February 5, 2012

Source Code Search Browser for Command-line

Here is a handy utility ( viewsrc.sh ) for searching through your existing code base for code examples.  Let say you are developing some new code and you need to add a feature but you can not quite remember the syntax you used before.   No problem...without even having to leave the code you are in you can search for that example.

Syntax:  viewsrc.sh  <searchstring>  <fileExtension>
Where: searchstring is a regular expression (one accepted by "egrep"
and fileExtension is the file extension of the files to look at... for example... "pl" or "c" or "sh" or "py" or "html", ... you get the idea.

Features:
  • Search-word can be a simple regular expression
  • Search within files of certain file extensions
  • The search is from the current directory and also within sub-directories
  • Search is not case-sensitive
  • Every search match results in the full contents of the matching file being appended to one temporary file.
  • The temporary file is opened with vi/vim in "view" read-only mode.
  • The cursor is placed at the first occurrence of the search-word.
  • Press "n" for next match, or "N" for previous match.
  • The filename where each match occurred, is placed at the top of each matching file that was appended to the temporary file.
  • No changes are made to original code, only to a temporary file.
  • The advantage of the "viewsrc.sh" utility vs grep is that you see the full context of the match by seeing the entire source file.
  • When you find a match you like, just go backwards in the file until you see the source file name, if you want to know which file the match occurred in.
Example 1:
You would like to find examples ofPerl programs that use the "foreach" keyword

viewsrc.sh foreach pl

You will be placed in view mode, and if there are any matches, just press "n" to go to the next match, "N" for previous match, and when you are done, just quit like you would in vi/vim ...

:q!

Example 2:
You are editing some Bash code...


vi mycode.sh

Without leaving your editing session, search for example code containing the word "case"


:!viewsrc.sh case sh

Press 'n' to view next occurrance of the word "case".
Then exit with ":q!".

Note:  If you wanted to you can search multiple times in succession.  For every search you do you would have to do a ":q!" to return to the previous environment.

Here is the source for Bash Shell:


#!/bin/bash
#Script:  viewsrc.sh
#Purpose: This script is used to search through the current directory
# and down for any files that contain the search word (param 1) within files
# with the given file extension (param 2).
#
# Every file that contains the searched for word is
# concatenated into one temp file and viewed with the cursor on the
# first occurence.
# Press "n" to see the next occurance of the searchword.
# You exit with the command    :q!
#
# Example:
# You are editing a file in vi and you want to see korn shell code examples
# using the word "grep".
# To view this examples do the following:
#    1)  Press ESC to make sure you are not in edit mode
#
#    2)     :!viewsrc.ksh grep ksh  <press enter>
#
#    3)  A read-only temporary file will be displayed with the cursor on
#        the first match of the searchword you searched for.
#
#    4)     n    to see the next match if one exists.
#           N    to see the previous match
#
#    5)  To quit the view session enter the following:
#           :q   <press enter>  or
#           :q!  <press enter>
#
#    6)  You will see  "[Hit return to continue]"
#                       <press enter>
#
#    7)  You are back where you started.
#
#    Note:  Step 2 can be done multiple times to drill down into
#           procedure calls.  Just do Step 5 to go back a level.
#           Press CTRL-G to verify what file you are currently in
#           (whether it is your original file or the temp file)
#
#    Note:  If you don't get a match you will see something like:
#           /tmp/srch.030130_095607" [Read only] No lines in the buffer
#                   Do step 5 to go back to your original file.

#Function Name :  fullpath()
#Purpose     : Accepts a filename
#              Returns the full pathname of the file
#-----------------------------------------------------------------
# NOTE:  fullpath function works relative to the current directory.
#        Either the full path or the basename will work ok.
#-----------------------------------------------------------------

#Function to construct a full pathname for the given filename
fullpath ()
{
s=$i                                    #s is a string containing the filename
ts=`echo $s  | awk '{print substr($0,1,1)}'`    #ts is the 1st char of string
if [[ "$ts" = "/" ]]                    #Is leading char a slash
then
  #No change is necessary because it is already a full path
  echo $s
else
  #Tack on current directory without the leading ./
  ts=`echo $s  | awk '{print substr($0,1,2)}'`          #1st two char of string
  if [[ "$ts" = "./" ]]                 # does it start with "./"
  then
    # Replace the dot with the current working directory and tack on
    # the string from the second character to the end of the string
    s=$(echo `pwd``echo $s | awk '{printf ("%s", substr($0,2))}'`)
    echo $s
  else
    #Just a filename was given so build a string from the current working
    #directory, a slash and then the filename
    s=$(echo `pwd`/`echo $s | awk '{printf ("%s", substr($0,1))}'`)
    echo $s
  fi
fi
}


#- - - MAIN SCRIPT STARTS HERE - - -
USAGE="\n\nUsage:  viewsrc.ksh searchword FileExtension\n"
if (($# != 2))                  #One parameter required
then
    echo $USAGE
    exit 1
fi

SEARCHWORD=$1

# Declare Variables
TIMESTAMP=`date +%y%m%d_%H%M%S`
MATCHFILES=/tmp/temp.$TIMESTAMP
SEARCHFILE=/tmp/srch.$TIMESTAMP
DIRFILE=/tmp/dirfile.$TIMESTAMP

#If the searchfile already exists remove it
if [[ -f $SEARCHFILE ]]
then
   rm $SEARCHFILE
fi

#Create an empty file
touch $SEARCHFILE

#If the dirfile already exists remove it
if [[ -f $DIRFILE ]]
then
   rm $DIRFILE
fi

#Create an empty file
touch $DIRFILE

# Find Files That Contain The Searchword in various subdirectories
find . -name "*.${2}" -print > $DIRFILE

#Note:  in between the brackets is a space and a tab character
cat $DIRFILE | xargs grep -i -l "${1}" > $MATCHFILES

for i in `cat $MATCHFILES`
do
    # Consolidate The Files That Contain The Searchword
    echo "===========================================" >> $SEARCHFILE
    FP=$(fullpath ${i})
    echo "File: `hostname`:${FP}" >> $SEARCHFILE
    echo "===========================================" >> $SEARCHFILE
    cat $i >> $SEARCHFILE
done

# Display the SEARCHFILE at the first Searchword
EXINIT='set ic'
export EXINIT
view +/$SEARCHWORD/ $SEARCHFILE

# Cleanup Temporary Files
rm $MATCHFILES
rm $SEARCHFILE
rm $DIRFILE

TIPS:

  • To search on more than one word, put a period between words.
    Like....      
    viewsrc.ksh Searching.on.four.words sh
  • Searching on special characters like brackets would need to be backslash escaped
    Like....
    viewsrc.sh if.\[\[ sh


Enjoy!