A customer e-mailed us with a report saying that suddenly their job started failing on a blank line after exiting a program. The blank line had always been there but it suddenly started to be a problem.
Here is the job stream source.
!job jtest01,mgr.acct,neil !comment blank line after exit from query !purge file1x !query exit !showjcw jcw !showjcw cierror !purge file1x !eojFails:
:JOB JTEST01,MGR.ACCT,NEIL.
Priority = DS; Inpri = 8; Time = UNLIMITED seconds.
Job number = #j15161.
TUE, FEB 9, 2010, 12:04 PM.
HP3000 Release: C.70.00 User Version: C.70.00
MPE/iX HP31900 C.39.06 Copyright Hewlett-Packard 1987.
All rights reserved.
STREAMED BY NEIL,MGR.ACCT (#S240) ON LDEV# 5
STREAM DATE: TUE, FEB 9, 2010, 12:03 PM
********************* CALVIN A400 *************************
*** This is a private system operated for Robelle's ***
*** company business. Authorization from Robelle is ***
*** required to use this system. ***
*** Use by unauthorized persons is prohibited. ***
************************************************************
:comment blank line after exit from query
:purge file1x
^
File "FILE1X.NEIL.ACCT" not found. No purge done. (CIWARN 383)
:query
HP32216N.03.18 QUERY/NM TUE, FEB 9, 2010, 12:04 PM
COPYRIGHT HEWLETT-PACKARD CO. 1976
exit
END OF PROGRAM
Missing colon before command name. (CIERR 981)
REMAINDER OF JOB FLUSHED.
CPU sec. = 2. elapsed min. = 1. TUE, FEB 9, 2010, 12:04 PM.
Doesn't fail?
:JOB JTEST01,MGR.ACCT,NEIL.
Priority = DS; Inpri = 8; Time = UNLIMITED seconds.
Job number = #j15163.
TUE, FEB 9, 2010, 12:09 PM.
HP3000 Release: C.70.00 User Version: C.70.00
MPE/iX HP31900 C.39.06 Copyright Hewlett-Packard 1987.
All rights reserved.
STREAMED BY NEIL,MGR.ACCT (#S240) ON LDEV# 5
STREAM DATE: TUE, FEB 9, 2010, 12:07 PM
********************* CALVIN A400 *************************
*** This is a private system operated for Robelle's ***
*** company business. Authorization from Robelle is ***
*** required to use this system. ***
*** Use by unauthorized persons is prohibited. ***
************************************************************
:comment blank line after exit from query
:purge file1x
^
File "FILE1X.NEIL.ACCT" not found. No purge done. (CIWARN 383)
:query
HP32216N.03.18 QUERY/NM TUE, FEB 9, 2010, 12:09 PM
COPYRIGHT HEWLETT-PACKARD CO. 1976
exit
:showjcw jcw
JCW = 0
:showjcw cierror
CIERROR = 383
:purge file1x
^
File "FILE1X.NEIL.ACCT" not found. No purge done. (CIWARN 383)
:eoj
CPU sec. = 2. elapsed min. = 1. TUE, FEB 9, 2010, 12:09 PM.
So the question is what is the difference between the two jobs. Why
does one fail and not the other?
The difference between the two jobs is actually Query. In the first instance where the job does fail is the run of query is done via a file command.
In the second instance where Query does not fail, the run of query is done via path resolution.
The tricky part of this is of course that their is no way to tell from the job stream.
You can use help to tell the difference:
Path Resolution or Implied run.
:help query PROGRAM FILE: QUERY.PUB.SYSCommand File / UDC
:help query USER DEFINED COMMAND FILE: QUERY.NEIL.GREEN run querynm.pub.sys
So we know that the difference in the two jobs is that the one that failed is the run of Query thru the command file. But Why?
Are blank lines Allowed?
Well first of all the trick here is to make sure a blank line is or isn't allowed. A quick test shows that any job with a blank line fails:
:comment blank line after comment Missing colon before command name. (CIERR 981) REMAINDER OF JOB FLUSHED. CPU sec. = 1. elapsed min. = 1. TUE, FEB 9, 2010, 12:40 PM.
So we know for certain that a blank line does cause an abort in a job stream.
With this information, the real question becomes why didn't the job with the blank line after the exit fail, but rather why didn't it fail before?
The answer is that the lines are never executed. As shown below note the lines after the exit.
!comment lines after exit !query exit Hi this is text after the run of query it would be surprising to know that this job still runs. !showjcw jcw !eojThese lines never show up in the job stream when it is streamed.
:comment lines after exit :query HP32216N.03.18 QUERY/NM TUE, FEB 9, 2010, 12:49 PM COPYRIGHT HEWLETT-PACKARD CO. 1976 exit :showjcw jcw JCW = 0 :eoj CPU sec. = 2. elapsed min. = 1. TUE, FEB 9, 2010, 12:49 PM.
What is happening is that in the job stream after the exit, the job doesn't really know where it is in the $stdin, the input commands to query, so it keeps reading until it sees the ! character.
This means that the lines after the exit and before the !showjcw are essentially thrown away. This re-orientation on where you are on the $stdin saves the job from failing on the blank lines or lines with garbage.
BASE mybase,5,pass G SHIP-DATA ITEM SHIP-DATE,DATE,MMDDYY IF (($STDDATE(SHIP-DATE)>= 20100101 AND $STDDATE(SHIP-DATE)<= 20100131)) OUT TEMPSHIP,LINK XWell the reason for this is that the DEFAULT value for "cutoff" on what century to assume is 10. Therefore if you date field is January 15,2010, but in the form 011510, the year portion being 10 will have an assumed century of 19, because the default value for the cutoff year is 10.
The solution to this is to issue the command:
set date cutoff 15or a similar number that will be greater than the oldest year of data that you want to have a 20 for the century.
Happily you do not need to add this command to every task but rather just add to your suprmgr files. On MPE this is suprmgr.pub.sys and on HP-UX this is /opt/robelle/suprmgr.
But Suprtool should definitely be able to copy the data *much* faster than MPE's FCOPY utility could.
The trick, of course, is getting a new NM KSAM file built correctly first. You could use a complicated BUILD command or you could do this:
:file n=newfile; disc=
That will create a new NM KSAM file called NEWFILE with the FLIMIT equal to the FLIMIT of your current CM file, and with all the keys set up correctly, but will copy only a single record into it. Then use Suprtool:
:suprtool >in oldfile >out newfile, erase >xeq
Finally, if it's a big KSAM file you may wish to create it as temporary first, then SAVE it after you're done, to squeeze even more speed out of the process:
:file n=newfile; disc=; temp <<-- Note the temp designation :fcopy from=oldfile; to=(*n); subset=0,1 :suprtool >in oldfile >out newfile, erase >exit :save newfile
But let's backup and discuss the functionality that MPE and the Suprtool list command could provide thru file equations as this is the main question that users ask, which is How do I get the output from the list command to either a file or a device.
On MPE you can either do:
file suprlist;dev=lpor
file suprlist=myfile;rec=-132;disc=100000;dev=disc
The first would send the output from the list command to the device LP and the second file equation would re-direct the output from the list command to a disc file. Well on HP-UX the concept of file equations doesn't exist. There is I/O re-direction but this can be cumbersome when you have to input the commands and filter the output so we specifically added options on the list command to allow for the same functionality.
A sample of an lp command that prints to a printer device called dragon:
lp -ddragon t1.cc request id is dragon-1387 (1 file)
lpstat -pwhich means to output all printers. From that output you can determine the device names for various printers. lpstat -d prints your default printer and lpstat -a prints out all devices that are accepting output requests, which may be your best bet on a system with a large LP subsystem. Keep in mind that this assumes that the CUPS system is installed and enabled. If you want to know more about CUPS you can start with the wiki page at:
http://en.wikipedia.org/wiki/CUPS
The bottom line is that you can use lpstat to determine the device printer names that are on your system.
>in bla4 >list device dragon standard >xeq request id is dragon-1388 (standard input) IN=272, OUT=272. CPU-Sec=1. Wall-Sec=1.Suprtool opens a pipe to the device dragon and sends the output from the list command to that device and uses the lp command/program to output the file. So essentially when you enter the command list device dragon, Suprtool is sending the output to lp with the device of dragon, as in:
/usr/bin/lp -ddragon.We also provide an environment variable to change the lp program from being used which by default is /usr/bin/lp to being whatever you to use specifying the ROBELLE_LP environment variable with the value of the other third party program:
export ROBELLE_LP=/usr/thirdparty/lp
>set list formfeed on >in bla4 >list file myreport standard >xeq IN=272, OUT=272. CPU-Sec=1. Wall-Sec=1. >in bla3 >list file myreport append standard >xeq IN=272, OUT=272. CPU-Sec=1. Wall-Sec=1.In addition with the set list formfeed on command you can put multiple reports in one file and have them print and do the page eject(s) between each report. So in the above example the list of bla4 would print out and then bla3 would print on a new page. As always if you have any feedback, questions or advice, please feel free to e-mail me at neil at robelle.com.
Is there any way to export the connections and other settings from QEDIT on one PC and load onto another?
Thanks.
Yes. The steps are very easy:
A customer called and asked how to find the difference between a Maturity date and Todays date.
First step, I change the matdate to be ccyymm and create a ccyymm for $today. Note that the Maturity date or matdate, is a dat field in the format of ccyymmdd and is in an I2 container.
>in mature
>form
File: mature (SD Version B.00.00) Has linefeeds
Entry: Offset
MATDATE I2 1
Entry Length: 4 Blocking: 1
>ext matdate=matdate / 100
>def todaydt,1,4,double
>item todaydt,date,ccyymm
>ext todaydt=$today
>out mature2,link
>xeq
IN=20, OUT=20. CPU-Sec=1. Wall-Sec=1.
So the data now looks like this:
>in mature2 >list >xeq >IN mature2 (0) >OUT $NULL (0) MATDATE = 200906 TODAYDT = 200907 >IN mature2 (1) >OUT $NULL (1) MATDATE = 200907 TODAYDT = 200907 >IN mature2 (2) >OUT $NULL (2) MATDATE = 200908 TODAYDT = 200907 >IN mature2 (3) >OUT $NULL (3) MATDATE = 200909 TODAYDT = 200907 >IN mature2 (4) >OUT $NULL (4) MATDATE = 200910 TODAYDT = 200907 >IN mature2 (5) >OUT $NULL (5) MATDATE = 200911 TODAYDT = 200907 >IN mature2 (6) >OUT $NULL (6) MATDATE = 200912 TODAYDT = 200907 >IN mature2 (7) >OUT $NULL (7) MATDATE = 201001 TODAYDT = 200907 >IN mature2 (8) >OUT $NULL (8) MATDATE = 201002 TODAYDT = 200907Now the final step converts the dates to a month number and calculates the difference. :
SUPRTOOL/UX/Copyright Robelle Solutions Technology Inc. 1981-2009.
(Version 5.2 Internal) WED, JUL 01, 2009, 8:12 PM Type H for help.
in mature2
ext matdate
ext todaydt
def mmdiff,1,4,double
ext mmdiff=(($truncate(matdate / 100) * 12) + matdate mod 100) - &
(($truncate(todaydt / 100) * 12) + todaydt mod 100)
list
xeq
>IN mature2 (0) >OUT $NULL (0)
MATDATE = 200906 TODAYDT = 200907
MMDIFF = -1
>IN mature2 (1) >OUT $NULL (1)
MATDATE = 200907 TODAYDT = 200907
MMDIFF = 0
>IN mature2 (2) >OUT $NULL (2)
MATDATE = 200908 TODAYDT = 200907
MMDIFF = 1
>IN mature2 (3) >OUT $NULL (3)
MATDATE = 200909 TODAYDT = 200907
MMDIFF = 2
>IN mature2 (4) >OUT $NULL (4)
MATDATE = 200910 TODAYDT = 200907
MMDIFF = 3
>IN mature2 (5) >OUT $NULL (5)
MATDATE = 200911 TODAYDT = 200907
MMDIFF = 4
>IN mature2 (6) >OUT $NULL (6)
MATDATE = 200912 TODAYDT = 200907
MMDIFF = 5
>IN mature2 (7) >OUT $NULL (7)
MATDATE = 201001 TODAYDT = 200907
MMDIFF = 6
>IN mature2 (8) >OUT $NULL (8)
MATDATE = 201002 TODAYDT = 200907
MMDIFF = 7
>IN mature2 (9) >OUT $NULL (9)
MATDATE = 201003 TODAYDT = 200907
MMDIFF = 8
>IN mature2 (10) >OUT $NULL (10)
MATDATE = 201004 TODAYDT = 200907
MMDIFF = 9
>IN mature2 (11) >OUT $NULL (11)
MATDATE = 201005 TODAYDT = 200907
MMDIFF = 10
>IN mature2 (12) >OUT $NULL (12)
MATDATE = 201006 TODAYDT = 200907
MMDIFF = 11
>IN mature2 (13) >OUT $NULL (13)
MATDATE = 201007 TODAYDT = 200907
MMDIFF = 12
>IN mature2 (14) >OUT $NULL (14)
MATDATE = 201008 TODAYDT = 200907
MMDIFF = 13
>IN mature2 (15) >OUT $NULL (15)
MATDATE = 201009 TODAYDT = 200907
MMDIFF = 14
>IN mature2 (16) >OUT $NULL (16)
MATDATE = 201010 TODAYDT = 200907
MMDIFF = 15
>IN mature2 (17) >OUT $NULL (17)
MATDATE = 201011 TODAYDT = 200907
MMDIFF = 16
>IN mature2 (18) >OUT $NULL (18)
MATDATE = 201012 TODAYDT = 200907
MMDIFF = 17
>IN mature2 (19) >OUT $NULL (19)
MATDATE = 201101 TODAYDT = 200907
MMDIFF = 18
IN=20, OUT=20. CPU-Sec=1. Wall-Sec=1.
Recently a customer asked how they could fix some of the Zip codes which were in the form of:
123450000
and are stored in a numeric J2 field.
The customer wanted to normalize those zip codes that had the four trailing zeroes to be 12345 instead of 123450000.
The first step we wanted to do was to determine all of the codes that needed to be updated and what the old zip would be and what the new zip would be.
>base membrs >get member-file >def new-zip,1,4,double >if zip > 99999 and (zip mod 10000) = 0 >ext account >ext zip >ext new-zip = zip / 10000 >list >xeq
So what is the above doing?
Well the if command looks at all records that are greater than 99999 and ends in the four zeroes, which is what the (zip mod 10000) = 0 is doing. This should isolate just those records that the customer wanted to fix!
Once we determined that we had the correct records selected we easily updated them with:
>base membrs >get member-file >if zip > 99999 and (zip mod 10000) = 0 >update >ext new-zip = zip / 10000 >list >xeq
One of the more recent questions that has come up lately is How Do I Import data with Suprtool. Let's say we have the following data layout, Image, Eloquence or SD file, it doesn't matter. The purpose of this small white paper would be to extract data to a ".csv" file, and then import that same data back. Closing the loop so to speak.
To begin our project we will extract the data from a sample data source and output to an SD file and then use STExport to create a CSV file.
>get d-inventory
>out dinv,link
>xeq
IN=13, OUT=13. CPU-Sec=1. Wall-Sec=1.
:run stexport.pub.robelle
$in dinv
$out dinvcsv
$xeq
In=13. Out=13. CPU-Sec=1. Wall-Sec=1.
$listf dinvcsv,2
ACCOUNT= GREEN GROUP= NEIL
FILENAME code ------------LOGICAL RECORD----------- ----SPACE----
SIZE TYP EOF LIMIT R/B SECTORS #X MX
DINVCSV 152B VA 13 13 1 16 1 *
So at this point we know have a file, typically how a customer would want
to import using Suprtool for adding to a database etc. Note that Suprtool
does not handle variable-length files so the first step is to convert the
Variable length file to fixed length.
On MPE you can convert using Fcopy:
/file dinvcsvf;rec=-152,1,f,ascii /fcopy from=dinvcsv; to=*dinvcsvf;newOn HP-UX you can use Qedit:
/t dinvcsv /set keep var off /k dinvcsvfOr Awk: (Thanks to Barry Lake from Allegro for fixify.sh)
#!/bin/sh
#
# Script to turn a typical bytestream file (variable length
# records) into a file with fixed length records by padding
# records with spaces as needed. The resulting record length
# will be that of the longest record in the file.
export INFILE=$1
OUTFILE=$(mktemp)
# Step 1: Find the length of the longest record in the input file
# file and store it in a variable for use in the next step.
export MAXLEN=$(awk 'BEGIN { len = 0; }
{ if (length > len)
len = length; }
END { print len; }' $INFILE )
echo Maximum record length in \"$INFILE\" is $MAXLEN.
# Step 2: Get the MAXLEN variable; use it to create a string
# of that many blanks; then use that to pad each input
# record as needed to make it the same length as the
# longest record.
awk 'BEGIN { "echo $MAXLEN" | getline maxlen; spaces = "";
for(i=0; i < maxlen * 1; i++)
spaces = " "spaces; }
{ print substr($0 spaces, 1, maxlen) }' $INFILE >$OUTFILE
# Step 3: awk can't edit a file in place so its output was
# written to a new file which we then pour back into the
# original file. Note: we could just as easily have done
# cp $OUTFILE $INFILE or mv $OUTFILE $INFILE, but in that
# case we'd lose the original file's creator and
# permission bits.
cat $OUTFILE > $INFILE
rm $OUTFILE
You can run the above script:
$./fixify.sh datafile
At this point Suprtool can now read what is essentially variable-length data in a fixed length file.
Now remember what a typical CSV file looks like! Byte fields are surronded by quotes and separated or delimited with commas:
159,19910827,1,50532001,5053,9449,"Test index","Test index","Test index","Test index"
The first step is to separate out each field from the record based on the delimeter, which in this case is the comma. I use byte lengths for each number field based on the rules for output ,ascii table.
Duplicated below:
I1 J1 06 bytes I2 J2 11 bytes I3 J3 16 bytes I4 J4 20 bytes K1 05 bytes K2 10 bytes Zn n+1 bytes Pn n bytes
in dinvcsvf
{define targets}
def bin-x,1,6
def last-x,1,11
def qty-x,1,11
def prod-x,1,9
def supp-x,1,9
def cost-x,1,8
def desc1-x,1,22 {Note room for quotes}
def desc2-x,1,22
def desc3-x,1,22
def desc4-x,1,22
{define source}
def record,1,152
{extract using split }
ext bin-x=$split(record,first,",")
ext last-x=$split(record,",",1,",")
ext qty-x=$split(record,",",2,",")
ext prod-x=$split(record,",",3,",")
ext supp-x=$split(record,",",4,",")
ext cost-x=$split(record,",",5,",")
ext desc1-x=$split(record,",",6,",")
ext desc2-x=$split(record,",",7,",")
ext desc3-x=$split(record,",",8,",")
ext desc4-x=$trim($split(record,",",9,last))
{where?}
out myfile,link
xeq
Now we just need to "close the loop" and extract the individual byte type fields into their appropriate data types. In this step we also "clean" the data of the double quotes.
Keep in mind that the data format that we want:
The defines for the above are in the section starting with the comment {Actual targets}
in myfile
{re-define number in display format}
def bin-z,bin-x,display
def last-z,last-x,display
def qty-z,qty-x,display
def prod-z,prod-x,display
def supp-z,supp-x,display
def cost-z,cost-x,display
{Actual targets}
{number}
def bin ,1,2,integer
def last,1,4,double
def qty ,1,4,double
def prod,1,8,display
def supp,1,8,display
def cost,1,4,packed
{bytes}
def desc1,1,20
def desc2,1,20
def desc3,1,20
def desc4,1,20
ext bin=$number(bin-z)
ext last=$number(last-z)
ext qty=$number (qty-z)
ext prod=$number(prod-z)
ext supp=$number(supp-z)
ext cost=$number(cost-z)
clean '"'
ext desc1=$trim($clean(desc1-x))
ext desc2=$trim($clean(desc2-x))
ext desc3=$trim($clean(desc3-x))
ext desc4=$trim($clean(desc4-x))
out loop,link
xeq
The SD file (loop) is now in the same format and layout of the original Dataset extraction prior to the STExport task. We've now closed the loop.
>form loop
File: LOOP.NEIL.GREEN (SD Version B.00.00)
Entry: Offset
BIN I1 1
LAST I2 3
QTY I2 7
PROD Z8 11
SUPP Z8 19
COST P8 27
DESC1 X20 31
DESC2 X20 51
DESC3 X20 71
DESC4 X20 91
Limit: 13 EOF: 13 Entry Length: 110 Blocking: 37
So to summarize the first step is to split out the data into separate fields with byte data types. Then to convert by re-defining with a new name to reference the data as display and use $clean and $number to extract into the final targets.
A customer recently asked how they could use Suprtool to generate some random data. The answer is you can't. But Barry Durand came up with a reasonable Unix shell script, using something he found on the net.
Here is a script that called rand3:
a[0]=a; a[1]=b; a[2]=c; a[3]=d; a[4]=e; a[5]=f; a[6]=g; a[7]=h a[8]=i;
a[9]=j; a[10]=k;a[11]=l; a[12]=m; a[13]=n; a[14]=o; a[15]=p a[16]=q;
a[17]=r; a[18]=s; a[19]=t; a[20]=u; a[21]=v; a[22]=w a[23]=x; a[24]=y;
a[25]=z; a[26]=1; a[27]=2; a[28]=3; a[29]=4 a[30]=5;
a[31]=6; a[32]=7; a[33]=8; a[34]=9; a[35]=0
echo ${a[$RANDOM%36]}${a[$RANDOM%36]}${a[$RANDOM%36]}
./rand3.sh gtj ./rand3.sh 7o2
Someone asked how to remove trailing spaces on HP-UX. Here is one way:
pr -t INFILE >INFILE.spaces
One of the more common tech calls we get is: my job aborted with Floating Exception(coredump) what is wrong?
A Floating point exception is an HP-UX specific arithmetic trap which occurs when the system encounters a character that it cannot deal with when converting to/from Floating Point. These errors occur in Suprtool and STExport for HP-UX, mainly because Suprlink just matches bytes, it doesn't try to transform the data in any manner. It is the transformation or coercion from one form to another where these FPEs occur. In Suprtool, it could be from nearly any type to nearly any other type. An FPE in Suprtool is typically a coercion error from one type TO floating point/display or packed.
One of two possibilities exist:
Most of the time we see that there is an incorrect definition of a field or record.
in baddisp,rec 80,lf def a,1,5,display >if a=65312 >ext a >out * >xeq 65312 IN=6, OUT=1. CPU-Sec=1. Wall-Sec=1.If we do essentially the same job, but get the record size wrong by one, we see the dreaded:
>in baddisp,rec 81,lf >def a,1,5,display >if a=65312 >out * >xeq Floating exception(coredump)
Remember that on Unix files are just a series of bytes, there is no record structure. So in the case above we read the first record and it lines up ok, however, the next record will start at the wrong spot and the coercion will fail.
more baddisp 01234 21222 00345 12345 54321xx 65312xx
Note that the last two records have characters right beside them so if we get the definition of the beginning display field incorrect we will end up trying to interpret the record with the 4321x as display data, which will of course fail.
Now we show an incorrect field definition:
>in baddisp,rec 80,lf
>def a,2,5,display {obviously wrong}
>if a<>65312
>out *
>ext a
>xeq
1234
1222
0345
2345
Floating exception(coredump)
You can see each record as it is read by using the out * command instead of output to some file. (Hint: Out * just means out to stdlist so it may not be useful if you have lots of fields). The List command to a file may be more useful, you could then tail the file to see where the problem was.
It is advisable at first to limit the number of records with the numrecs command while investigating these errors. Typically if a record definition and/or field definition are incorrect, then the abort will occur in the first few records so numrecs 10 command could save you waiting for a long listing to go to the screen.
more baddisp 01234 21222 00345 12345 54321xx 65312xxIn the first step below we just read the data and put into a self-describing file:
>in baddisp,rec 80,lf >def a,2,5,display >ext a >out mydata,link >xeq IN=6, OUT=6. CPU-Sec=1. Wall-Sec=1.Now in a later step we may reference the bad field and the abort will then occur. So sometimes you have to trace back a step to the one that created the file you are reading:
>in mydata
>form
File: mydata (SD Version B.00.00) Has linefeeds
Entry: Offset
A Z5 1
Entry Length: 5 Blocking: 1
>if a<>65213
>list
>xeq
>IN mydata (0) >OUT $NULL (0) A = 1234? >IN mydata (1) >OUT $NULL (1) A = 1222? >IN mydata (2) >OUT $NULL (2) A = 345? >IN mydata (3) >OUT $NULL (3) A = 2345? Floating exception(coredump)
Note that the "?" is a clue that you have a bad definition as Suprtool can handle spaces after numbers, but it aborts when it encounters the "x". Hopefully this gives you information on what a Floating exception is.
We are working on catching SIGFPE and trying to give more information about the circumstances in a future version of Suprtool, but for now the above are some of the techniques that you would likely need to use to find the bad data or bad definitions.
I've been working on projects that requires that I have two copies of some of our source code modules and I want to compare all the files for the given groups to monitor the changes.
Here is an MPE/Qedit command file that will compare all the files in two groups. Keep in mind that this is dependent on the groups having the same number of files in the group and the same filenames.
parm group_one group_two
purge fg1,temp >$null
purge fg2,temp >$null
purge fgout,temp >$null
echo Comparing !group_one against !group_two
listf @.!group_one,6 >fg1
listf @.!group_two,6 >fg2
build fgout;rec=-256,1,f,ascii;disc=20000;temp
file fgout;acc=append
/set totals off { No totals listed }
/tq fg1,text
/set length 256
/set right 256
/appendq "~" @
/addq last=fg2
/lsortq all
/glueq "~" @
/changeq 1 "compare " @
/changeq "~"," @
/changeq " ,"," @
/changeq " ,"," @
/changeq " ,"," @
/changeq " ,"," @
/changeq " ,"," @
/changeq " ,"," @
/changeq " ,"," @
/appendq ",tempout >$null" @
/appendq "~:if compareoutcount <> 0 then" @
/divideq "~" @
/changeq "~"" @
/appendq "~:fcopy from=tempout; to=*fgout >$null" "compareoutcount"
/divideq "~" @
/changeq "~"" @
/appendq "~:endif" "fgout"
/divideq "~" @
/changeq "~"" @
/useq *
reset fgout
/textq fgout,yes
/listq $r @
/set totals on
The command file essentially builds a list of compare commands and then
if differences are found in the files, then the compare output is copied
to a single file, which is listed on my attached printer at the end.
This is specific to my needs but is easily customizable. Feel free to send any comments or improvements to neil@robelle.com.
By default, Suprtool typically puts the temporary space it needs on /var/tmp. If this volume runs out of space when sorting data you may receive an error similar to this:
Error: Failure in ROBSORT'INPUT routine ROBSORT'INPUT Error 12 Unable to write to the RSORTSCR file (length = 27657) No space left on device Error: ROBSORT'END Failed ROBSORT'END Error 19 File system error Error 0You can control what directory/volumeset that Suprtool puts its temp sort scratch files by setting the TMPDIR variable prior to running Suprtool.
export TMPDIR=/home/bigsortspaceThe above will cause Suprtool to create Suprtool temp file in the directory space /home/bigsortspace.
Some customers have asked recently how to install the latest version of Suprtool without making it the production version.
An example of customers that are looking for this are some of the Ecometry customers that are migrating off of the HP 3000 and don't want to upgrade their production version of Suprtool or STExport but they want to use the clean features available in either Suprtool or STExport for migrating data off the 3000.
This HowDoI article will address these issues in the following manner:
The following instructions are ONLY to give a method for users on MPE to install the program files from Suprtool to TEST or use new features. This can be done by anyone that wants to test the latest version of Suprtool, STExport and Suprlink.
The steps to do this are:
:hello manager.sys,pub :file t=stprod;dev=disc :restore *t;@nm.pub.robelle;create;show=short :purge stprod :stream robelle.job.robelleWait for the job to send you a completion message:
========= Robelle.Pub.Sys ================= = Robelle account structure ready. ============================================
:hello mgr.robelle,pub :Run extend.pub Please enter the Product Verification code : n Please enter the Verification code : n Please enter the Second Verification code : n
Extend will cycle thru all the files and patch the files. You may get an error message such as:
Error: Unable to open SUPRTOOL.PUB.ROBELLE EXCLUSIVE VIOLATION: FILE BEING ACCESSED (FSERR 90)
This message just means that someone is running the program that extend tried to open this is not a problem unless it is one of the three files that need to be patched by this process which are:
suprnm.pub.robelle
linknm.pub.robelle
stexpnm.pub.robelle
Once you have applied the codes you should be able to run Suprtool/STExport on their own.
run suprnm.pub.robelle
run stexpnm.pub.robelle.
When you restore the files you will get six files restored, but the other files will not impact anything. Please note that the methods above will not load the latest help files, which is typically what most customers want, and since the latest manuals are now available on line, it makes the most sense.
Again this is for only those customers that want to try just the Suprtool, Suprlink and STExport binaries.
The main reason for most Ecometry customers to install the latest version is to get the clean features of either Suprtool or STExport.
While the clean features of each product do the same thing, they are implemented in slightly different ways.
When STExport cleans data, it just very simply removes all offending characters from each byte field, for all those characters specified in the clean command. STExport takes an SD file and creates a "PRN" file so to speak, but it allows you to control more about the file being created, such as delimiters and format of the numbers etc.
For example if you wanted to remove any extraneous Tab characters in byte fields the stexport task would be:
$ in mysdfile $clean "^9" $set cleanchar " " $out myexport $xeq
You can have the Clean function clean the field, and instead of replacing with a space, STExport will essentially shift characters to the left by Setting the CleanChar in the following manner:
>Set Cleanchar “”
STExport will pad the field that was cleaned with the appropriate amount of characters with a space at the end of the field.
STExport has other easy to use features such as Clean SPECIAL which automatically cleans all byte type fields with characters ranging from Decimal 0 to Decimal 31.
The Suprtool implementation of Clean is more focused and will allow you to actually fix the Source data as opposed to fixing the data "along the way", as STExport does.
Suprtool will "clean" or replace all of the characters specified in the Clean command from a byte type field when invoked by the $Clean function. To define what characters that need to be replaced you use the clean command with the character you want to clean in quotes. Since most of the characters that you will need to clean are unprintable, you can enter the decimal equivalent of the character. This is denoted by entering the "^" character in quotes preceding the decimal number of the character you wish to clean.
An example of how easy it would be to clean your database of certain "bad" characters in byte-type fields would be as follows:
>base mydb,1,;
>get customer
>clean "^9","^10","^0","^7"
>update
>ext address(1) = $clean(address(1))
>ext address(2) = $clean(address(2))
>xeq
The SPECIAL keyword automatically defines Clean characters of Decimal 0 thru to Decimal 31.
>base mydb,1,;
>get customer
>clean special
>update
>ext address(1) = $clean(address(1))
>ext address(2) = $clean(address(2))
>xeq
The above method cleans the source data in the database and allows you to fix any data in the database prior to exporting. This gives you a little more flexibility.
Now you can also, ONLY fix the data that has clean characters, (which makes much more sense):
>base mydb,1,;
>get customer
>clean special
>if $findclean(address(1)) or $findclean(address(2))
>update
>ext address(1) = $clean(address(1))
>ext address(2) = $clean(address(2))
>xeq
So the findclean function will look for any of the characters specified by the clean command and only those records will be updated.
|
|---|