Vector Length

Most computations in Radioss, like element or contact forces, are performed by packets of MVSIZ. This parameter is adjustable to match so-called vector length. This parameter is also important for cache locality. It is optimized by parallel development team according to hardware characteristics

New treatments need to respect this programming model which is to split the loop over number of elements or nodes by packets of MVSIZ. This will ensure optimal vector length and cache size as well as minimal local storage (local arrays of size MVSIZ instead of number of elements or nodes)

Loop Control

Loop Size

Inside a loop it is recommended to keep the number of instructions reasonable. 20 instructions or less is good. Very long loops should be split to keep cache efficiencyMost compilers will be able to fuse short loops, while they will probably fail to vectorize long complex loops

Calling a procedure inside a loop inhibits vectorization

Data Dependency

The loop below is not vectorized due to possible dependence (same value of INDEX(I) for different I): 

DO I = 1, N
         K = INDEX(I)
         A(K) = B(K)
         B(K) = 2*A(K)
END DO

In case of no true dependence, vectorization needs to be forced by adding a compiler directive

To keep portability across different platforms and compilers, an architecture specific include file exists named vectorize.inc that manages vectorization directives. The programmer just needs to add this include file just before the DO loop:

#include "vectorize.inc"
       DO I = 1, N
            K = INDEX(I)
            A(K) = B(K)
            B(K) = 2*A(K)
       END DO

Notice there is another include file named simd.inc which makes unconditional vectorization, even if a true dependence is detected by the compiler. It is recommended to only use vectorise.inc which is more conservative regarding correctness

For Intel compiler:

• vectorize.inc corresponds to !DIR$ IVDEP

• simd.inc corresponds to !DIR$ SIMD

Procedure CALL

Calling a procedure inside a loop inhibits vectorization therefore it is not authorized

Inside Radioss, there are vectorized versions of procedures, basically the loop is put inside the procedure rather than outside:

• VALPVEC to replace JACOBIEW

• VINTER to replace FINTER

Nested Loops

In practice, only the inner most loop will be vectorized. So the inner most loop needs to be the largest one.

For fixed size loops it is possible to unroll them by hand or to use Fortran90 enhancement. Then the compiler is able to vectorize the outer loop

Example Nested Loop with NEL >> DIM

DO I = 1, NEL
    DO J=1, DIM
      A(I,J) = B(I,J) + C(I,J)
    END DO
END DO 

To be transformed to:

DO J=1, DIM
    DO I = 1, NEL
     A(I,J) = B(I,J) + C(I,J)
    END DO
END DO 

Or using Fortran90 notation:

DO I=1,NEL
    A(I,:DIM)=B(I,:DIM) + C(I,:DIM)
END DO

Or for this simple case:

A(:NEL,:DIM)=B(:NEL,:DIM) + C(:NEL,:DIM)

Power

Never use real variable for integer power because of the cost of real power arithmetic. Take

care to not use real variable defined in constant.inc when integer is enough

Div

For invariant, it is advised to multiply by invert instead of doing a division by a constant inside a loop

Fortran90 Array Operations

Use of Fortran90 array operations is encouraged as long as code readability is kept , by always specifying array bounds to avoid confusion between variable and array arithmetic.

 Example:

INTEGER, DIMENSION(NUMNOD) :: A, B, C
A = B + C

    ! confusion between variable and array operation

                               ! confusion between variable and array operation  
A(:NUMNOD) = B(:NUMNOD) + C(:NUMNOD)

    ! default lower bound:1