Tip

The Lazy Coder: Communicate with programs using data queues

The problem

You want a program to wake up IMMEDIATELY when you need it to do something, such as print an invoice, FTP a file or send an e-mail, then go back to sleep until you ask for something else. There are lots of reasons for doing this, such as separating out a complex set of business rules or simply keeping complex programs using their own sets of memory. Whatever you're doing, you want to communicate with that program.

So, what do you do? Here are four methods to consider:

Use a data area
OK, that's not a big deal, but it is a limited way to accomplish your goal. You can only tell that program to fire up and start with one thing to do or with a limited set of things to do. You can't give that program any more details other then a "one at a time" sort of thing or you will need relatively complex programming to determine what to do (such as reading a data area's 5000 bytes and breaking that up into individual records). The program will ALWAYS wait a designated amount of time and then start up.

The advantage of this is this will be completely backed up every time the data area object is saved. The disadvantage: There are limitations with locking and timing.

Use a logical file selecting SPECIFIC statuses
OK. It's backed up because this is data-written to a file. However, you're still waiting a designated amount of time until the program to wakes up, which isn't cool sometimes. When you're creating the logical file, selecting a specific status to react to is a GOOD way to scan the file, but you may want to position the status you'd like to use or react to and read it. That helps maximize the use for that logical file instead of specializing it. More system resources would be wasted for a single-purpose file. But on the down side, you're still waiting for something to happen before you can proceed.

Use a data queue
OK. It's NOT backed up. Once the thing is read, it's read. The data can be sorted in First in/First out, Last in/First out or something called "Keyed" where you receive SPECIFIC stuff from the queue. The record doesn't miraculously reappear, however, when you shout, "Hey! That was a software problem!" (I know, I tried.) But there are a lot of pluses to our current situation.

The function to CHECK the data queue says essentially, "Wait for an entry to appear" or "Wait a certain number of seconds for something." (The only problem is if the something you want NEEDS to get done as soon as it arrives. Then what?)

Data queues are also FAST, and they are multi-system capable. You can read a data queue from a PC (which works AWESOME), and there are ways to have an AS/400 send to another AS/400 using something called a DDM queue. (That's a topic for a later conversation.)

Use a data queue AND a log file
Now you've got it. Program the data queue to wait for an entry. If the data queue times out (or really the delay expires), check the file you're going to create to KEEP all these queue entries in.

So, you create a status of, say, 1 when received and not processed, 2 when processed, and 3 when completed (simple, no?). You check this file JUST TO SEE if you have any status 1 entries. It's the same program, but it's processing things from the file instead. This is the methodology I use with a PC reading a data queue from an AS/400 application. The PC wakes up AS SOON as something comes into the data queue. If it can't find it, then it looks for status 1 log entries. Process those AS IF THEY CAME IN FROM THE QUEUE. They are OLDER entries that weren't processed due to some calamity -– software glitch, communications issue, etc. When they are DONE being processed, change the status 1 item to a status 2. It's simple, and you have a backup when there were difficulties. Change that record ONLY WHEN YOU KNOW it's done. If a process fails in the middle, you know EXACTLY where to start off.

How do you get started?

First: Understand that a data queue is kind of like a spool queue. The printer can print only one job at a time. It prints the reports in a first in, first out order.

Still not clear? Imagine a stack of pancakes -- you can eat ONLY one at a time, and you must always eat the one on the top. Every new pancake delivered from the kitchen goes on the bottom of your stack. So when reading from the queue, you eat the TOP pancake (YUM!). When sending to the queue, you lift up the stack of pancakes and place your newest pancake on the bottom. (Are you getting hungry to work on some data queues? I am. Or I may want breakfast -- I'm not sure.)

Any bit of information can be placed in the queue -- data or "messages." This information can be parsed or split up with a data structure -– whatever you'd like -- but just KEEP IT SIMPLE. Example: You SEND an ORDER NUMBER to the queue. When the program READS the queue, it prints the invoice for that Order Number. Business Logic is in one place, and you're using intracommunication to talk to your programs.

Create the data queue

CRTDTAQ DTAQ(&LIB/&DTAQ) TYPE(*STD) MAXLEN(&LEN) SIZE(*MAX2GB) AUTORCL(*YES) TEXT('Data Queue For ...')
Where 
          TYPE = is *STD or *DDM -- a *STD Data queue is a LOCAL data queue. 
               *DDM queues can send their info to a remote SNA device.
          MAXLEN = Length you'd like to use -- so you can send whole sets of 
               parameters to your programs using data queues.
          SIZE = a Number (who wants to guess that!), *MAX16M -- or 16 megs -- 
              Older Data Queues were SMALL, *MAX2GB. The size can expand (AND 
              CONTRACT) to 2Gigs maximum. This is a VERY efficient use of storage.
          AOTURCL(*YES) = Since data queues USE storage, they contract and 
              expand every time they are used. Normally you'd need to perform a 
              Reclaim Storage to recover this. For anyone still nostalgic about the 
              old days, set this to *NO. If not, then use *YES.

There are FAR MORE parameters, but concentrate on those first. Take the defaults on the rest of the parameters for now. Keep in mind that you can also set up a PC program to read this data queue, sending parameters you'd like the PC program to use (as I mentioned earlier).

How do you get stuff from the queue?

CALL       PGM(QRCVDTAQ) PARM(DTAQ LIB LEN MSG WAIT)  
Where
          DTAQ = Name (10 Byte Alpha) 
          LIB = Library (10 Bytes Alpha) 
          LEN = LENGTH of transmission you're receiving (15 Numeric - 5 decimals) 
                When Field Length is GREATER then zero, then you can determine if 
                your wait time expired and perform tasks based on an expired wait 
                time, such as "check the Queue Log file for activity." 
          MSG = Here's the data being sent -- you can also create the data queue 
                to give you the user IDs of who sent the data to the queue. (Define 
                this as the length of the queue being used. So, if you defined 1024, 
                define this as 1024 bytes.)
          WAIT = Here's the special part of a data queue. When this is set to a -1, 
                 the queue waits until there's an entry on it. OK, cool. BUT if you 
                wanted to wait, say, 30 seconds, set this to 30. (Wait is in seconds!)

How do you get stuff INTO the queue?

Send to the data queue

CALL       PGM(QSNDDTAQ) PARM(DTAQ LIB LEN MSG)
Where 
          DTAQ = The Name of the queue (10 Bytes) 
          LIB  = The Library where the queue exists (10 Bytes) 
          LEN = The Length of the stuff being put into the queue. Trimming this
                   information to the size of the data being inserted will cut down on
                   the OVERALL size of the queue.
          MSG = What yer' Sendin'
 

Hey! I want a do-over. I put too much junk into the queue while I was testing.

CALL PGM(QCLRDTAQ) PARM(DTAQ LIB)
          DTAQ = The name of the queue (10 Bytes) 
          LIB = The library where the queue exists

Next month I'll go into the flows of what this means and where you can utilize this. Where I work we have multiple systems, and this methodology can send stuff to other systems or other systems can read the queue. I use the logging method so that I'm not relying on programs being up all the time or functioning properly all the time. All it takes is a single HALT in a process AFTER you've read a queue entry.

-----------------------------------------
About the author: Andrew Borts is webmaster at Seta Corp., a jewelry catalog company based in Boca Raton, Fla. He is often a frequent speaker at COMMON and past president of The Southern National Users Group, an AS/400 user group based in Deerfield, Fla.

==================================
MORE INFORMATION
==================================

  • Working with *DTAQ in RPGLE A user asks, "How can I send or receive data from *DTAQ?" Search400.com expert John Brandt had some advice.
  • File type that works like a message queue
    Another user asks, "Is there a file type on the iSeries that works like message queues? For example, while one program (or many programs) are writing to the bottom of the file, another program is independently reading from the top of the file. The data is presented to the read in FIF0 order." Search400.com expert John Brandt has a suggestion.
  • .iF3uacVGNyO.3@.ee84636/4697>Data area locked
    A user was having a problem with a data area locking, but "TSCHMITZ" had a few ideas to help him out.


This was first published in February 2004

There are Comments. Add yours.

 
TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

Disclaimer: Our Tips Exchange is a forum for you to share technical advice and expertise with your peers and to learn from other enterprise IT professionals. TechTarget provides the infrastructure to facilitate this sharing of information. However, we cannot guarantee the accuracy or validity of the material submitted. You agree that your use of the Ask The Expert services and your reliance on any questions, answers, information or other materials received through this Web site is at your own risk.