Parsing iso8583 messages Part 2

Today we will be showing how to extract the various components out of an iso string.
The combination of a tcp server as well as code to extract the iso message components and send it for further processing is what is popularly called an interface server in payment processing .
Specifically the components we are interested in are

message type indicator also known as mti
bitmap
data elements

The code snippets used for this post is in this git repository. Also the 1993 ascii protocol will be used here as well as erlang programming language for the code explanations . So git clone the repository,cd to iso_process_tut_repo and fire up your erlang vm and lets begin !!.

Consider the message below

1200F230040102B000000000000004000000

1048468112122012340000100000001107221800

000001161204171926FABCDE123ABD06414243

000termid1210Community106A5DFGR1112341234234

the iso message in this case is an ascii string(erlang has no concept of strings just a list of unicode points). The function below is what processes the messages and returns a map containing the various fields of the message as well the mti and bitmap

-spec process_message([{list | binary,pos_integer()}])->map().

process_message({binary,Rest})->

the first statement converts the string to a binary. Binary strings or binaries store data in a more space efficient manner and has more efficient functions for handling large binary data.

Bin_message = erlang:list_to_binary(Rest),%%this gives <<“01581200F230040102B…”>>

the next step is to find the mti which is made up of the first 4 digits.We do that and then store the mti along with with some metadata in a map. the ?MTI_SIZE is macro which is defined at the top of the file and is equal to 4.

Mti = binary:part(Bin_message,0,?MTI_SIZE),%%this gives <<“1200”>>
Mti_Data_Element = maps:from_list([{ftype,ans},{fld_no,0},{name,<<“Mti”>>},{val_binary_form,Mti}]),

%%this gives #{fld_no => 0,ftype => ans,name => <<“Mti”>>,val_binary_form => <<“1200”>>}

the next step is to get the bitmap which is a hexadecimal(base 16 number)which starts from the fifth digit and is either 8 or 16 in size.
The bitmap tell us which data elements exist in the message. the bitmap is either a primary bitmap or a secondary bitmap. the first bit in a bitmap tell us whether the secondary bitmap exists. the bitmap is either an 8 byte(primary) or 16 byte(secondary) hexadecimal which when converted to base 2 shows the presence or absence of fields from 1 to 128. Each bit in a primary/secondary bitmap shows the presence/absence of a data element.
Each digit when converted into base 2 and left padded to a max of 4 digits with zero represents the presence or absence of 4 data elements. For example the bitmap 4210001102C04804 is 0100001000010000000000000001000100000010110000000100100000000100 in base 2.
this means Fields 2, 7, 12, 28, 32, 39, 41, 42, 50, 53, 62 are present and there is no secondary bitmap since bit 1 is 0.
Find below code sections for getting the bitmap out of our message as explained above from

Fthdig = binary:part(Bin_message,4,1),%%this gives F

Bitmap_transaction = << << (convert_base_pad(One,4,<<“0”>>)):4/binary >> || <One:1/binary> <= Bitmap_Segment >>, %%this gives <<“111100100011000000000100000000010…”>>

the next three statements creates a map which contains the bitmap and the mti .
The next step is to get the data elements which is the actual content of the message itself and contains info like

transaction amount
currency
transaction type

the start_index calculation tells us to start processing the binary string at a particular position.Since the primary bitmap is always sent with every message and the secondary bitmap is optional we start the processing from the point of the secondary bitmap .
Of importance to note is the get_spec_field function which contains info about every data element.The function returns info such as

maximum field length
field type
whether field is fixed length or variable length
header length if field has a header

Variable length fields have a header which shows how long the field is even though the field may have a maximum length .
You could say this is the erlang representation of the iso93 asci spec.This also makes it quite easy to modify the spec if some fields change.

the fold_bin function is a custom function which kind of works like a list fold. In this case instead of a list it takes a binary,accepts a fun for processing the binary and processes the binary until the binary is of size zero .there is also an accumilator variable in there just like with a list fold .

fold_bin(_Fun, Accum, <<>>) -> Accum;
fold_bin(Fun, Accum, Bin) ->
{NewBin, NewAccum} = Fun(Bin, Accum),
fold_bin(Fun, NewAccum, NewBin).

The reason for this is because we will be passing the bitmap as the input element with the iso message also being passed as an input but as an accumulator in this case.The bitmap of course represents the presence or absence of a field .
If a field is present then we extract the data element out of the iso message because the iso message will be available on every iteration and then move on.If the field is not present we move on to the next element anyway .
The fold_bin segment which gets the data elements and puts them in a map works like this in its simplified form

OutData=fold_bin( fun
(Bitmap, {Data_for_use_in,Index_start_in,Current_index_in,Map_out_list_in}) when X =:= <<“1”>> ->
in this case field exists
Data_for_use_in = Iso Message as input in fold_bin function
Index_start_in = Position we are in in iso message which corresponds to a specific start position for data element
Current_index_in = current data element
Map_out_list_in = accumulator for data elements
statement to extract data elements based on field length,maximum length etc follow..
{Rest_bin,{Data_for_use_in,New_Index,Fld_num_out,NewMap}};
%%output which contains NewMap which contains the data element
Rest_bin = bitmap which contains presence/absence of data elements yet to be processed
Data_for_use_in = Iso Message as input in fold_bin function
New_Index = Position we are in in iso message which corresponds to a specific start position for next data element
Fld_num_out = next data element

Also New_Index is the position of the next element so we know where in the binary to start processing the next date element and Fld_num_out is the fld_num which we will use for getting the spec for a field using the get_spec_field function

(Bitmap, {Data_for_use_in,Index_start_in,Current_index_in,Map_out_list_in}) when X =:= <<“0”>> ->
in this case field does not exist
…statement to extract data elements based on field length,maximum length etc..
{Rest_bin,{Data_for_use_in,New_Index,Fld_num_out,NewMap}};

end, {Bin_message,Start_index,1,Map_Data_Element},Bitmap_transaction),
%% Bin_message = Iso Message
Start_index = is used to tell where to start processing the binary from ,
1 = this tells us we are starting from field 1
Map_Data_Element = the accumulator for the data elements where each data element is added this accumulator as a map with some metadata about the data element also added

At each iteration as we are going through the iso message there is a calculation which is done to find out metadata about the data element . This metadata will give us the header length of the field or if even the field has a header .After we get the header or not we then get to calculate the actual message and its starting and ending points .
After the data element is obtained it is put in the output map along with some metadata about the data element.

Please browse through the source code to better understand the code as well as how to get the various data elements if my explanation is not enough .
There is also a function on how to do the above with strings.

process_message({list,Rest})->

As i said earlier binaries occupy less space and has more efficient functions for processing but you can browse through the code to get a better idea of processing the message but using string functions .
To test it out by sending sample iso messages open the erlang shell and compile the iso_process with c(iso_process).You can then send iso messages by first starting the server with

iso_process:start_iso_server().

afterwhich you can send iso messages with

iso_process:send(“01581200F230040102B00000000000000400000010”

“48468112122012340000100000001107221800000001161204171926FA”

“BCDE123ABD06414243000termid1210Community106A5DFGR1112341234234”).

Note that the iso message must be preceded by length of the message so in this case 0158 was the length of the message and was added to the message when the message was being sent.

Alternatively there is a java file Test_java_client.java which you can use to edit and send messages to the iso server example.
After editing the various iso fields in the file you can then compile the file using

javac -cp .:jpos-2.0.7-SNAPSHOT.jar Test_java_client.java

and send the message to the erlang iso server which should have been started using

java -Xbootclasspath/p:jpos-2.0.7-SNAPSHOT.jar Test_java_client 8002

This is where i wind down.