Is there a file type on the AS/400 that works like message queues? While one program (or many programs) are writing to the bottom of the file, another program is independently reading from the top of the file. The data is presented to the read in FIF0 order. The result of the read is to delete the record from the file. Another need would be to have the reading program wait on a record to show up without having to loop. Kind of like a read to a terminal -- hang a read and wait for the enter key to be pressed to satisfy the read and return control to the program. In the IBM documentation, I have found data queues work this way, but the manual says they should not be allowed to get very large (less than 100 records). Also, in the IBM documentation, I have found references to MQSeries, but I need a solution that does not require another licensed product. Do you have any other solutions more creative that a DB2/400 table.
If you want to stay with a file structure, I would suggest you use a database trigger program. You can set a program to be called based on a write to any AS/400 file. The trigger time (before, or after the action has completed), the operation (Insert, Update, Delete...) is passed as well as the value(s) of the record. Your program could decide what to do based on which action and data being passed and act accordingly. I have used Data Queues for many years and have not had a problem exceeding the number of records in a data queue. As long as the job(s)/program(s) to process the data queue entries are sufficient to keep the number of records low, you should be safe. In addition, when you create your data queue(s), you have the option of forcing them to disk. The maximum record length for a data queue is 64,512 characters, and I haven't seen a number of records recently, but I regularly have them in the 100,000 range.