SAS Enterprise Miner 6.1: C and Java Score Code Basics

®

TM

SAS Enterprise Miner 6.1 C and Java Score Code Basics

® The correct bibliographic citation for this manual is as follows: SAS Institute Inc. 2009. SAS Enterprise TM Miner 6.1: C and Java Score Code Basics. Cary, NC: SAS Institute Inc.

SAS® Enterprise MinerTM 6.1: C and Java Score Code Basics Copyright © 2009, SAS Institute Inc., Cary, NC, USA All rights reserved. Produced in the United States of America. For a hard-copy book: No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, or otherwise, without the prior written permission of the publisher, SAS Institute Inc. For a Web download or e-book: Your use of this publication shall be governed by the terms established by the vendor at the time you acquire this publication. U.S. Government Restricted Rights Notice: Use, duplication, or disclosure of this software and related documentation by the U.S. government is subject to the Agreement with SAS Institute and the restrictions set forth in FAR 52.227-19, Commercial Computer Software-Restricted Rights (June 1987). SAS Institute Inc., SAS Campus Drive, Cary, North Carolina 27513. 1st electronic book, December 2009 ® SAS Publishing provides a complete selection of books and electronic products to help customers use SAS software to its fullest potential. For more information about our e-books, e-learning products, CDs, and hard-copy books, visit the SAS Publishing Web site at support.sas.com/publishing or call 1-800727-3228. ®

SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Other brand and product names are registered trademarks or trademarks of their respective companies.

iii

Contents Chapter 1 C and Java Score Code in SAS Enterprise Miner 1 SAS Enterprise Miner Tools That Produce C and Java Score Code 2 SAS Formats Support 4 Generated C and Java Code 5 Generated C Code 5 DB2 User-Defined Functions 5 C Code Usage 7 C Formats Support 8 C Formats Support Distribution 9 C Formats Usage 9 Generated Java Code 10 Java Code Usage 11 Java Scoring Jars 11

Chapter 2 Scoring Example 13 Create Folders for the Example 13 Gather Files 13 Create Enterprise Miner Process Flow Diagram 15 Scoring with C Code 15 Scoring with Java Code 19

Chapter 3 Notes and Tips 23 Missing Values 23

iv

Contents

Chapter Error! No text of specified style in document.

Appendix 1 Programming Information 25 General Code Limitations 25 Supported Functions 26 Supported SAS Operators 27 Conditional Statement Syntax 28 Variable Name Length 28 Character Data Length 28 Extended Character Sets 28

Appendix 2 Example Java Main Program 29 Appendix 3 Example C Main Program 31 Appendix 4 SAS System Formats Supported Java Scoring 35 Appendix 5 SAS System Formats Supported for C Scoring 41 Appendix 6 C Compiler Command Examples 43 C Compiler Command Examples 43

C H A P T E R

1

C and Java Score Code in SAS Enterprise Miner

SAS Enterprise Miner Tools That Produce C and Java Score Code ........................................ 2 SAS Formats Support ................................................................................................................. 4 Generated C and Java Code ....................................................................................................... 5 Generated C Code ........................................................................................................................ 5 DB2 User-Defined Functions ...................................................................................................... 5 DB2 Data Types............................................................................................................ 7 C Code Usage ............................................................................................................................... 7 C Formats Support ...................................................................................................................... 8 C Formats Support Distribution ......................................................................................... 9 C Formats Usage .................................................................................................................. 9 Generated Java Code................................................................................................................. 10 Java Package Name........................................................................................................... 11 Java Code Usage ....................................................................................................................... 11 Java Scoring Jars...................................................................................................................... 11 Java Scoring Jars Distribution ........................................................................................ 12 Java Scoring Jars Usage ................................................................................................... 12 SAS System Formats ......................................................................................................... 12

Analytical data mining models generate score code that can be applied to new data in order to evaluate candidates for some defined event of interest. The model scoring code can exist in any number of programming languages. SAS Enterprise Miner generates model scoring code not only in SAS code, but for most models, in C and Java programming languages as well. Generating model score code in programming languages like C and Java provides greater flexibility in organizational deployment. Data mining score code in C and Java can be combined with source code and binary files that are distributed with Enterprise Miner, and then compiled for deployment in external C, C++, or Java environments. Experienced C, C++, or Java programmers can use this feature to extend the functionality of new and existing software by embedding the power of SAS Enterprise Miner analytical model scoring. It should be emphasized that creating a scoring application is a very complex and highly advanced task that requires expertise in several areas. The likelihood of successfully implementing a scoring system that incorporates SAS Enterprise Miner-generated C or Java code is exactly proportional to your fluency and experience with the environment in which you choose to implement your application. Testing of both the application and the generated code are critical to the success of any such project.

2 SAS Enterprise Miner 6.1: C and Java Score Code Basics

SAS Enterprise Miner Tools That Produce C and Java Score Code SAS Enterprise Miner can generate C and Java score code for most analytical models that are built from nodes that produce SAS DATA step scoring code. The following SAS Enterprise Miner nodes produce SAS DATA step scoring code, and are therefore able to create C and Java score code:

Sample Tools C and Java Generated Filter Node

No C or Java score code Append Data Partition File Import

nc

nc

Input Data nc Merge Sample

nc

Time Series

nc

Explore Tools C and Java Generated

No C or Java score code

Cluster SOM/Kohonen Variable Clustering Variable Selection

Association DMDB nc Graph Explore Market Basket Multiplot nc Path Analysis Stat Explore nc Text Miner

nc

Modify Tools C and Java Generated Impute Interactive Binning Principal Components Replacement Rules Builder ** Transform Variables **

No C or Java score code Drop

nc

Chapter 1: C and Java Score Code in SAS Enterprise Miner 3

Model Tools C and Java Generated AutoNeural Decision Tree Dmine Regression DMNeural Ensemble*** Gradient Boosting LARS Model Import Neural Network Partial Least Squares Regression Rule Induction Two Stage

No C or Java score code MBR

Utility Tools C and Java Generated End Groups Start Groups

No C or Java score code Control Point Merge Metadata nc Reporter nc SAS Code

nc

Credit Scoring Tools C and Java Generated Credit Exchange Interactive Grouping Scorecard nc

No C or Java score code Reject Inference

nc

Tool produces no score code

** It is possible to create code that cannot be correctly generated as C or Java. When you are creating transformations or expressions in the Transformation tool or the Rules Builder, careful inspection and testing is required to make sure your C and Java score code is generated correctly. *** Depends on members of process flow diagram


Enterprise Miner Score node can produce DATA step, C, and Java score code for most modeling process flow diagrams. However, a process flow diagram will not produce C or Java score code if the diagram includes a node that produces SAS code and also contains PROC statements or DATA statements. Enterprise Miner process flow diagrams that contain a node that is not listed in the above table will not generate C or Java code. The Enterprise Miner SAS Code node is a special case because it is an openended tool for user-entered SAS code. Enterprise Miner does not translate user-entered SAS code into C and Java score code. When Enterprise Miner encounters a model process flow diagram that includes the SAS Code node, it attempts to generate C and Java score code for the remaining portions of the process flow diagram. For the portion of the process flow diagram represented by the SAS Code node, Enterprise Miner inserts a comment in the generated C and Java score code that indicates the omitted input. For example, the comment in generated C code might resemble the following: /*--------------------------------**/ /* insert c code here

*/

/* datastep scorecode for emcode

*/

/* is not supported for conversion */ /*--------------------------------**/

In some cases, it might be possible for you to insert your own C code to take the place of the omitted SAS Code node content. Enterprise Miner also does not generate Java score code for SAS Code node content. When Enterprise Miner encounters a SAS Code node while generating Java code, the omitted code from the SAS Code node is replaced in the generated Java code with a call to a specific method. Enterprise Miner produces source code for an empty stub method with that specific name. This might enable you to substitute your own Java code to take the place of the omitted SAS Code node content.

SAS Formats Support SAS formats are functions used in the SAS System to configure the size, form, or pattern of raw data for display and analysis. There are two basic types of SAS formats: the pre-defined formats that are supplied with all SAS Systems, and the formats that are defined in score code. The formats that are defined in the score code are also referred to as user-defined formats. The user-defined formats that are used in the generated C and Java score code are supported primarily in the generated code. The SAS System formats for Java are supported through libraries distributed with SAS Enterprise Miner. The SAS System formats for C are supported through libraries distributed as the SAS Standalone Formats.


Generated C and Java Code The C and Java code that Enterprise Miner generates is a conversion of the algorithms and operations that the SAS DATA step code performed in the process flow diagram. It is important to understand that it is possible to use the tools available in SAS Enterprise Miner to generate valid SAS score code that cannot be correctly generated as C or Java score code. Any generated code should be tested thoroughly before deployment. The generated C and Java code represents only the functions that are explicitly expressed in the SAS DATA step scoring code. The C score code that Enterprise Miner generates conforms to the “ISO/IEC 9899 International Standard for Programming Languages – C.” The Java code that Enterprise Miner generates conforms to the Java Language Specification, published in 1996 by Addison-Wesley. While generated C or Java scoring code from an Enterprise Miner analytical model can be used as the core analytics for a scoring system, you should not confuse the generated C or Java scoring code with a complete scoring system. In either C or Java languages, the programs that you write to enclose the scoring code must provide a suitable environment for performing the data analysis. After you successfully run an Enterprise Miner model process flow diagram that generates C or Java score code, you can export your model as an SPK file that contains the generated C and Java code, or you can use the File menu in the results browser to save individual files.

Generated C Code The C scoring code is generated as several output files. They include the following: •

Cscore.xml is the XML description of the model that produced the code and the generated C code. It is valid XML. No Document Type Definition (DTD) is supplied.

•

DB2_Score.c is C code for DB2 scalar User Defined Functions for each of the output variables defined in the scoring code.

•

Score.c is the model score code generated as a C function. It is C source code and must be compiled before it can be executed.

DB2 User-Defined Functions In addition to generating the scoring algorithms developed in Enterprise Miner models, the C scoring component generates the C code for IBM DB2 user-defined functions. IBM user-defined functions, or UDFs, are tools that you can use to write your own extensions to SQL. The functions that are integrated in DB2 are useful, but do not offer the customizable power of SAS


Analytics. The Enterprise Miner-generated UDFs enable you to greatly increase the efficiency, versatility, and power of your DB2 database. The key advantages of using UDFs are performance, modularity, and the objectoriented UDF process. The UDF code that Enterprise Miner generates is matched to each specific model’s training data and the C scoring functions that are associated with the model. The UDF code that Enterprise Miner generates is only one of several ways to create score code in DB2. The generated source code for the UDFs is simple but expandable. The comments in the UDF source code contain templates of SQL commands that need to be registered in order to invoke the generated UDFs. Enterprise Miner can generate score functions that return values that are not useful in a scoring context. The UDFs that Enterprise Miner generates for a specific model are limited to the functions that return scoring output values that are considered to be of interest heuristically. The names of the scoring output variables are created by concatenating a prefix (for each type of computed variable) with the name of the corresponding target variable (or decision data set). Enterprise Miner produces UDFs for scoring output variables that begin with the following prefixes: D_ EL_ EP_ I_ P_

decision chosen by the model expected loss of the decision chosen by the model expected profit of the decision chosen by the model normalized category that the case is classified into predicted values and estimates of posterior probabilities

Enterprise Miner also produces UDFs for scoring output variables with the following names: _NODE_ _SEGMENT_ _WARN EM_CCF EM_CLASSIFICATION EM_DECISION EM_EVENTPROBABILITY EM_EXPOSURE EM_FILTER EM_LGD EM_PD EM_PREDICTION

tree node identifier segment or cluster identifier indicates problems with computing predicted values or making decisions average credit cost factor value fixed name for the I_ variable fixed name for the D_targetname variable fixed name for the posterior probability of a target event average exposure value identifies filtered observations average loss given default value average predicted value fixed name for the predicted value of an interval target


EM_PROBABILITY

EM_PROFITLOSS EM_SEGMENT SCORECARD_BIN SCORECARD_POINTS SOM_DIMENSION1 SOM_DIMENSION2 SOM_ SEGMENT

fixed name for the maximum posterior probability associated with the predicted classification fixed name for the value of expected profit or loss fixed name for the name of the segment variable bin assigned to each observation total score for each individual identifies rows in a Self Organizing Map (SOM) identifies columns in a SOM identifies clusters created by a SOM

Most of the code in the UDFs that Enterprise Miner generates is designed to handle the conversion of data types and missing values before and after the score function is called. The first function in the generated UDF code (load_indata_vec) is invoked by all the UDFs in the file in order to load the input data vector for the score function. Current DB2 code documentation states that each reference to a DB2 function (UDF or built-in) is allowed to have arguments that number from 0 to 90. The limitation on the number of arguments for each reference is a critical limitation for data mining jobs where even simple models can require hundreds of values. Enterprise Miner is capable of producing UDF code that contains more than 91 arguments, but DB2 cannot use any of the additional arguments.

DB2 Data Types The UDFs that Enterprise Miner generates accept only two SQL data types: DOUBLE and VARCHAR. Most databases use more than two SQL data types, so you should use care when you convert your DB2 data types for UDF calls in your code. DB2 provides functions that you can use to convert most data types to DOUBLE or VARCHAR. Another way to handle additional SQL data types in training data and score data is to perform the required data type conversions during the extract, transfer, and load (ETL) step of your data preparation. You can also modify the UDF source code that Enterprise Miner generates in order to convert data types for scoring.

C Code Usage To compile, link and run the SAS Enterprise Miner-generated C code you will need to first gather the tools, libraries and files required. The generated C code conforms to the “ISO/IEC 9899 International Standard for Programming Languages – C” so any current compiler should be able to compile the code.


Other than the standard C libraries the generated C code will have dependencies on the SAS Standalone Formats libraries. See the SAS Formats section below for details. The generated C code also depends on three C header files. • • •

cscore.h csparm.h jazz.h

The cscore.h and csparm.h files are distributed with the Enterprise Miner Server Windows systems. They are located in SASROOT\dmine\sasmisc. For UNIX systems they are located in SASROOT/misc/dmine. Copy them to your development environment. The cscore.h file has several operating system specific definitions that will, in most cases, need to be modified for your target operating system. Those modifications are documented in comments in the header file and in the following example. The jazz.h header file is distributed with the SAS Standalone Formats product. See details below in the C Formats Support section. Copy it to your development environment. To run the generated C code, you will need to create a main program to invoke the score function. You can view the score.c file and inspect it to determine how the score function should be called. The metadata file Cscore.xml also describes the generated function and its arguments. By default, the generated function accepts two pointers as arguments. The first argument points to an array of input data values. The second argument points to an array of output data values. The memory, which is required for each data value must be allocated by the calling program. Both arrays must be composed of the PARM data structure that is defined in the csparm.h header file. Each array element must contain either a double or a char *. The length of the memory referenced by each char* can be extracted from the Cscore.xml or by inspection of the original training data set. If the appropriate memory for each character value is not allocated before calling score(), the results are undefined. The position of each data value in its array can also be extracted from the XML or inferred from the #defines for the variable names found in the generated C code. These variable names are usually taken directly from the training data or derived from names in the training data. Such a main program can be as simple as the code in Appendix 3.

C Formats Support The SAS Enterprise Miner-generated C scoring code supports the use of SAS System Formats through the SAS Standalone Formats product. The SAS Standalone Formats do not depend on a SAS System environment in any


way. The SAS Standalone Formats product contains a header file and a set of libraries that will be needed for compilation, linking and running the Enterprise Miner-generated C scoring function. The SAS Standalone Formats are a set of link and run-time libraries compiled for each supported operating system. Those include the following: • • • • • • • • •

Windows 32-bit Windows Itanium 64-bit Windows x64-bit Solaris 64-bit AIX 64-bit Linux 32-bit Linux 64-bit HP-Itanium 64-bit HP 64-bit

C Formats Support Distribution The SAS Standalone Formats are distributed as downloads from the SAS Customer Support Web site. Look in the Knowledge Base section for Samples and SAS Notes. Search for Note 35872, titled “SAS Standalone Formats for SAS Enterprise Miner C Score Code.” Follow the instructions to download the package for your target operating system.

C Formats Usage The Enterprise Miner-generated C scoring function/application is linked to jazxfbrg so that at run-time the code in jazxfbrg can dynamically load the rest of the routines needed to support the SAS System formats. While only jazxfbrg might need to be present when linking the function/application, all of the files must be available at run-time. For the Standalone Formats, dynamic loading is accomplished through calls to standard System routines. Dynamic loading is an advanced topic in any C environment. The exact procedures, options, and environment variables used in compiling, linking, and running dynamically loaded code are different for every compiler, linker, and operating system. For example, on Windows, shared libraries are loaded from the environment variable PATH. This environment variable must be set to contain the directory path for the Standalone Formats shared libraries (jazwf*). The value of this environment variable must be the fully qualified directory name for the directory that holds the Standalone formats. On Solaris systems the Standalone Formats are dynamically loaded from shared libraries via the environment variable LD_LIBRARY_PATH. HP/UX systems use a slightly different environment variable, SHLIB_PATH. A thorough understanding of your target system’s procedures for compiling, linking and running with dynamically loaded code is required to successfully exploit the Standalone Formats and the code that is generated by the SAS Enterprise Miner C Scoring component.


For environments where the Standalone Formats support is not available or not desired, it should be possible for an experienced C programmer to modify the source code in the cscore.h header file that is distributed with SAS Enterprise Miner in SASROOT/dmine/sasmisc. The object of the modifications is to remove the dependency on the Standalone Formats and to support any format they want, with their own C code. If you write your own formats functions, you can integrate those functions into the logic that handles formats in cscore.h. The cscore.h file that is distributed with SAS Enterprise Miner already contains two examples of such C formatting code -- partial support for $CHAR and BEST formats. If your situation allows you to accept the limitations of those examples (no padding for $CHAR, and no scientific notation for BEST), you can use the example formats without any modification, and you can add any additional formats that you might need. In that case, any SAS Enterprise Minergenerated C scoring code that contains only those formats, and is compiled with the cscore.h header file will support those formats. In cases where Standalone Formats support is not desired, the dependency on the Standalone formats support can be removed. This can be accomplished by modifying a copy of the cscore.h header file that is distributed with SAS Enteprise Miner in SASROOT/dmine/sasmisc. In the cscore.h file, the preprocessor symbol FMTLIB is set to 0, which disables support for the SAS Standalone formats.

Generated Java Code Java scoring code is generated in several output files. The primary model logic is generated as a java class file. The other files are generated as Java source files, and the model’s variable metadata is encoded as XML. The Enterprise Miner-generated Java code is compatible with the version of Java that SAS Enterprise Miner uses. The generated Java files might include some or all of the following: •

DS.class is the actual DATA step code generated directly to Java binary code; there is no Java source code supplied.

•

DS_UEXIT.java will be generated only if code from unsupported tools was omitted from the generated Java code. This Java source code is a template that the customer can use to provide their own code for the omitted tool or node.

•

Jscore.xml is an XML description of the model that produced the code and the generated Java code. It is valid XML. No DTD supplied.

•

JscoreUserFormats.java is the Java source code that supports any user-written formats that might be used in the model. It is Java source code and must be compiled before it can be executed.


•

Score.java is the Java source code that implements the interface to DS.class. It is Java source code and must be compiled before it can be executed.

After you run an Enterprise Miner modeling flow, there are a number of ways to export the contents of your model along with the generated C and Java Scoring code. See Exporting the Results and the Score Node in the SAS Enterprise Miner Reference Help.

Java Package Name The Enterprise Miner-generated code contains an assigned package name. The package name effectively becomes the first part of the absolute class name. When compiling Java source code with a package name the Java compiler (javac) will search for the source and class files related by the package name in a path relative to the current working directory. The Java compiler uses the package name to form a hierarchical path for each related file. If for example, the package name has the default of “eminer.user.Score," the Java compiler will search for the package's files in the path eminer\user\Score. In order to compile the generated Java source code, all of the generated Java files (Jscore.xml not required) must be placed in a directory or folder tree that looks like the package name "eminer.user.Score." The default package name can be changed in the SAS Enterprise Miner Client before the flow is run, by selecting from the main menu Options Preferences, and then filling in a package name of at least 2 levels.

Java Code Usage SAS Enterprise Miner-generated Java scoring code depends on the classes and methods that are distributed as the SAS Enterprise Miner Java Scoring jars. In order to compile, or run Java score code generated by SAS Enterprise Miner, you need to copy the supporting Java archives and configure your system to make the jars available to Java.

Java Scoring Jars The Enterprise Miner Java Scoring jars support the classes and methods used in the generated java code including the use of SAS formats. The Enterprise Miner Java Scoring jars are: • dtj.jar • icu4j.jar • sas.analytics.eminer.jsutil.jar • sas.core.jar • sas.core.nls.jar • sas.icons.jar • sas.icons.nls.jar • sas.nls.collator.jar • tkjava.nls.jar


These jar files include support for most, but not all of the SAS System formats. The list of supported Java formats is detailed in Appendix 4.

Java Scoring Jars Distribution The SAS Enterprise Miner Java scoring jars are distributed as part of the SAS Enterprise Miner Server image. On Windows systems, they are found in the path SASROOT\dmine\sasmisc. For UNIX systems, check the path SASROOT/misc/dmine. It is recommended that you save copies of your Java scoring jar files in your scoring environment.

Java Scoring Jars Usage Wherever you want to compile and run the SAS Enterprise Miner-generated Java code, you will need to make the SAS Enterprise Miner Java scoring jars available to Java. Adding the directory path that contains your Java scoring jar files to your CLASSPATH environment variable enables both the compile and execution steps.

SAS System Formats Supported SAS System formats are listed in Appendix 4.

C H A P T E R

2

Scoring Example

Create Folders for the Example ................................................................................................ 13 Gather Files................................................................................................................................ 13 Create Enterprise Miner Process Flow Diagram ..................................................................... 15 Scoring with C Code .................................................................................................................. 15 Save and Edit C Code Component Files ........................................................................... 16 Organize C Code Component Files ................................................................................... 17 Compile, Link, and Run C Score Code in UNIX .............................................................. 17 Scoring with Java Code ............................................................................................................ 19 Save and Organize Java Code Component Files ............................................................. 20 Create Java Main Program ............................................................................................... 20 Compile and Run Java Score Code in UNIX ................................................................... 22

The scoring code that Enterprise Miner produces is affected by the choice of the data mining nodes that you use in your Enterprise Miner process flow diagram, by the sequence of the nodes in the process flow diagram, and by the data that you use to train your model. Likewise, changing the configuration of node settings in a process flow diagram, or modifying the variable roles, structure, or size of the training data set can change the generated scoring code. The score code that Enterprise Miner generates can be unique for every process flow diagram. The following example is for illustrative purposes and is not intended to be deployed as a real application. The example includes sections for producing both C and Java score code. The example score code is generated using an Enterprise Miner client on a Windows system. After the score code is created, it is extracted. Then the extracted score code is moved to a Solaris system, where it can be compiled and run.

Create Folders for the Example This example uses a number of folders or directories that you will need to create on your Enterprise Miner client. The example assumes that you will create the folders c:\temp\scorecode, c:\temp\scorecode\cscore, and c:\temp\scorecode\jscore.

Gather Files For the C scoring example, locate the SAS Standalone Formats for the system on which you will be scoring. In this example we will run the EM generated code on a Solaris system that will require the safmtss64.tar file. The “C


Formats Support Distribution” section of this document contains additional details about locating the SAS Standalone Formats. 1. Copy the tar file to the temporary cscore folder that you created for this example, C:\Temp\ScoreCode\cscore. 2. Locate the \sasmisc folder that is created when the Workspace Server for Enterprise Miner is installed. The default path for the sasmisc folder on a Windows Workspace Server for Enterprise Miner installation is C:\Program Files\SAS\SASFoundation\9.2\dmine\sasmisc. 3. Copy two files, cscore.h and csparm.h, from the \sasmisc folder to the C:\Temp\ScoreCode\cscore Folder. 4. For the Java scoring example, locate the folder in your Workspace Server for Enterprise Miner installation that contains the Enterprise Miner Java Scoring .jar files. The default folder location in UNIX is !SASROOT/misc/dmine. The default folder location on Windows systems is C:\Program Files\SAS\SASFoundation\9.2\dmine\sasmisc.

5. Copy the Enterprise Miner Java Scoring .jar files from the installation source folder to the the local folder that you created at the beginning of this example: C:\Temp\ScoreCode\jscore

Chapter 2: Scoring Example 15

Create Enterprise Miner Process Flow Diagram 1.

Launch Enterprise Miner, create a new project, and in your new Enterprise Miner project, create a new diagram.

2. Use the Enterprise Miner Toolbar shortcut button for Create Data Source to open the Data Source Wizard. Use the Data Source Wizard to specify the sample SAS table SAMPSIO.DMAGECR, and then use the wizard’s Advanced Advisor setting to configure the SAMPSIO.DMAGECR variable good_bad as the target variable. Keep the wizard’s default settings for the rest of the variables, and then save the All: German Credit data source with the data set role of Train. 3. Drag your newly created German Credit Data data source from the Data Sources folder of the Projects panel to the diagram workspace. 4. Drag an Interactive Grouping node from the Credit Scoring tab of the node toolbar to the diagram workspace, and connect it to the German Credit data source node. Leave the Interactive Grouping node in its default configuration. Note: The Interactive Grouping node is located in the Credit Scoring tab of the node toolbar in Enterprise Miner 5.3. If you are using Enterprise Miner 5.2, the Interactive Grouping node is located in the Modify tab of the node toolbar. 5. Drag a Regression node from the Model tab of the node toolbar to the diagram workspace, and connect it to the Interactive Grouping Node. Use the Selection Model property to configure the Regression node to perform Stepwise selection. 6. Drag a Score node from the Assess tab of the node toolbar to the diagram workspace, and connect it to the Regression node. Leave the Score node in its default configuration. 7. Right-click the Score node, select Run, and then select Yes in the confirmation dialog box to run your newly constructed process flow diagram. The C scoring code and the Java scoring code that Enterprise Miner generates are handled differently. Depending on which type of score code you intend to compile and deploy, your next steps are provided in either the section on Scoring with C Code or in the section on Scoring with Java Code.

Scoring with C Code The C scoring code that you generate with Enterprise Miner process flow diagrams can be compiled in most modern C or C++ development environments. The compilation results will vary, depending upon the compiler and its option settings. For example, some compilers produce warning messages about data type conversions because the compiler interprets data type conversion as a generic risk. Each compiler environment is different, and the range of option settings that are available through different compilers can generate different results. It is up to you, the score


code programmer, to properly configure and investigate your chosen compiler settings and warnings.

Save and Edit C Code Component Files 1. When your Enterprise Miner process flow diagram run completes, select the Results button in the Run Status window. 2. From the main menu in the Results window, select View Score Code to open the C Score Code window.

Scoring

C

3. In the C Score Code window, ensure that the list box at the bottom of the window is set to Scoring Function Metadata. 4. From the Results window main menu, select File Save As and save the file as cscore.xml in the c:\temp\scorecode\cscore directory that you created at the beginning of this example. 5. In the C Score Code window, return to the list box at the bottom of the window and change the setting from Scoring Function Metadata to Score Code. 6. From the Results window main menu, select File Save As and save the file as score.c in the c:\temp\scorecode\cscore directory that you created at the beginning of this example. 7. For the Solaris SPARC architecture, change the value used for missing values. Search the cscore.h file for the text string, “#define MISSING”. You should find a line that looks like this: #define MISSING

WIN_LE_MISSING

Edit this line so that it reads as follows: #define MISSING UNX_BE_MISSING Each operating system has its own value for missing. Cscore.h must be modified for each system. See Missing Values for more details. Change the defined value of SFDKeyWords to be blank. Some systems require special directives to correctly store the function name in an objects export table. The cscore.h header file provides a macro called SFDKeyWords for those systems. By default, the SFDKeyWords macro is configured for the Windows directive. For systems other than Windows, or if you are creating an object other than a DLL, you will need to modify the SFDKeyWords macro. If your situation does not require any directives, change your #define statement for the SFDKeyWords macro to define SFDKeyWords as blank. Search your cscore.h file for the string, “SFDKeyWords”. You should find a line that resembles the following: #define SFDKeyWords

extern

__declspec( dllexport )


Edit the #define SFDKeyWords statement so that it reads as follows: #define SFDKeyWords then save your changes and close the cscore.h file.

Organize C Code Component Files 1. Create a main program to invoke the score function. For this example, such a main program can be copied from Appendix 3. Name the main program file csbasic.c and copy it to C:\temp\ScoreCode\cscore. 2. In your HOME directory on the target UNIX System, create a directory named /example. 3. In your new /example directory, create a subdirectory called \cscore. 4. Copy or FTP all of the following files to your /example/cscore directory: c:\temp\scorecode\cscore\safmtss64.tar c:\temp\scorecode\cscore\csparm.h c:\temp\scorecode\cscore\cscore.h c:\temp\scorecode\cscore\Score.c c:\temp\scorecode\cscore\Cscore.xml Most FTP clients will take care of the carriage-returns in Windows text files. If not, most Solaris systems provide a dos2unix command that you can use to handle carriage returns. The dos2unix command is usually found in the /bin directory.

Compile, Link, and Run C Score Code in UNIX All steps in this section are performed in the UNIX operating system. 1. Navigate to your UNIX example/cscore directory and unpack the SAS Standalone Formats tar file by submitting the following command: tar xf safmtss64.tar The tar process creates the example/cscore/safmts directory, which contains the SAS Standalone Formats files. 2. Copy the C header file jazz.h from the /safmts directory to the folder that you created for this example, ~HOME/example/cscore. 3. The SAS Standalone Formats routines are dynamically loaded from some of the files in the /safmts folder. In the Solaris operating environment, you can use the LD_LIBRARY_PATH environment variable to modify the path that is searched for dynamically loaded code. The LD_LIBRARY_PATH environment variable is read at process start-up, and is a colon-delimited list of locations to include in the load library search path. To include your newly extracted /safmts directory in your Solaris load library path, enter this command:


LD_LIBRARY_PATH=.:$HOME/example/cscore/safmts 4. After you set the library path environmental variable, export the setting so that it is visible to your child processes. Enter this command: export LD_LIBRARY_PATH 5. If GNU C Version 3.2.3 (SPARC-SUN-Solaris 2.8) is available, it is usually installed in /usr/local/bin/. You can use GNU C to compile and link your C scoring program with a single command. The link and compile command might resemble the following: gcc -m64 -ansi -I$HOME/example/safmts csbasic.c Score.c –lm $HOME/example/safmts/jazxfbrg -m64 -ansi -I csbasic.c Score.c -lm jazxfbrg

selects the 64-bit environment turns off the features of GNU C that are incompatible with ANSI C specifies the path for the jazz.h header file, which is part of the formats support. the main C program to be compiled the C scoring code that was generated by Enterprise Miner specifies the math link library an object library from which the SAS Standalone Formats objects are linked

The link and compile command above should produce a single executable file called a.out. 6. To execute the main program, submit the following code: a.out The output from running the executable file a.out should resemble the following: $ a.out >> First observation... csEM_CLASSIFICATION = GOOD csEM_EVENTPROBABILITY = 0.8710097610 csEM_PROBABILITY = 0.8710097610 cs_WARN_ = >> 4th observation... csEM_CLASSIFICATION csEM_EVENTPROBABILITY csEM_PROBABILITY cs_WARN_ $

= BAD = 0.4103733144 = 0.5896266856 =


Scoring with Java Code Enterprise Miner can generate Java source code and binary class files. You must have access to a Java development environment in order to be able to use the Java code that you generate with Enterprise Miner. The Java code distributed with and produced by Enterprise Miner was developed with the Java SE Development Kit 6. You can obtain a Java Developer’s Kit (JDK) from Sun at http://java.sun.com/.


Save and Organize Java Code Component Files 1. Run the example Enterprise Miner process flow diagram. When the run completes, click the Results button in the Run Status window. 2. Select View Scoring Java Score Code from the main menu of the Results window. The Java Score Code window opens. 3. In the Java Score Code window, ensure that the list box at the bottom of the window is set to Scoring Function Metadata. 4. From the Results window main menu, select File Save As and save the file as JScore.xml in the c:\temp\scorecode\jscore directory that you created at the beginning of this example. 5. In the Java Score Code window, return to the list box at the bottom and change the setting from Scoring Function Metadata to Score Code. 6. From the Results window main menu, select File Save As and save the file as Score.java in the c:\temp\scorecode\jscore directory that you created at the beginning of this example. 7. In the Java Score Code window, return to the list box at the bottom and change the setting from Score Code to User-defined Formats. 8. From the Results window main menu, select File Save As and save the file as JscoreUserFormats.java in the c:\temp\scorecode\jscore directory that you created at the beginning of this example. 9. Save the Java Class file as DS.class in the c:\temp\scorecode\jscore directory that you created at the beginning of this example.

Create Java Main Program 1. Determine the package name of the generated Java code. One way to determine the package name is to view the Jscore.xml file and look up the Java class name. In this example, the class name is eminer.user.Score.Score. Remove the last qualifier "Score" from the class name, and the remainder eminer.user.Score is the package name. 2. You must provide a Java main program. The Java main program needs to instantiate the generated scoring class, provide input data, invoke the score method, and handle the scoring outputs. Appendix 2 contains an example Java main program. For this example, save the code in Appendix 2 as Jsbasic.java, and move the Jsbasic.java file to the folder that you created at the beginning of this example, c:\temp\scorecode\jscore. 3. On the UNIX system where the score code will be deployed, create the following directory structure in your HOME directory: $HOME/example/jscore/eminer/user/Score


4. FTP or copy all the .jar files from the Windows folder c:\temp\scorecode\jscore to the UNIX folder that you created, $HOME/example/jscore. 5. FTP or otherwise copy all the .java and related .class files from the c:\temp\scorecode\jscore folder to the UNIX folder at $HOME/example/jscore/eminer/user/Score. 6. When finished, the list of files in the $HOME/example/jscore directory should resemble the following: $ ls -1 dtj.jar eminer icu4j.jar sas.analytics.eminer.jusutil.jar sas.core.jar sas.core.nls.jar sas.icons.jar sas.icons.nls.jar tkjava.jar tkjava.nls.jar

7. The contents of the $HOME/example/jscore/eminer/user/Score directory should resemble the following: $ ls –l DS.class Jsbasic.java JscoreUserFormats.java Score.java

Note: Windows .java files will need to have the carriage-returns in the body removed. Many FTP clients will automatically remove carriage-returns in text files. If not, most Solaris systems provide a dos2unix command to perform that task. The dos2unix command is usually found in the UNIX /bin directory.


Compile and Run Java Score Code in UNIX All steps in this section are performed on the UNIX operating system. 1. Set the CLASSPATH environment variable to contain absolute paths for the .jar files that are distributed with Enterprise Miner Java Scoring. One way to set environment variables is at a prompt, on a single line, enter something that resembles the following: export CLASSPATH=.:$HOME/example/jscore/dtj.jar:$HOME/example/j score/sas.analytics.eminer.jsutil.jar:$HOME/example/jsco re/sas.core.jar:$HOME/example/jscore/sas.core.nls.jar:$H OME/example/jscore/sas.icons.jar:$HOME/example/jscore/sa s.icons.nls.jar:$HOME/example/jscore/sas.nls.collator.ja r:$HOME/example/jscore/sas.icons.jar:$HOME/example/jscor e/tkjava.nls.jar:$HOME/example/jscore/icu4j.jar 2. Your current working directory should be the parent directory of the package tree. The parent directory of the package tree in the example is $HOME/example/jscore. Invoke the Java compiler on the source files using a command that resembles the following: javac eminer/user/Score/*.java 3. The result should be a set of newly-created Java class files that implement the Jscore interface. 4. After you compile the Jsbasic main program and the Enterprise Miner generated source code, copy the Jsbasic.class file from $HOME/example/jscore/eminer/user/Score to the working directory you want to use to deploy the Java scoring code. 5. To execute your Java scoring code program, on the command line enter this code: java Jsbasic 6. The output from your Java scoring code program should resemble the following: >> First observation... EM_CLASSIFICATION = GOOD EM_EVENTPROBABILITY = 0.8710097609996974 EM_PROBABILITY = 0.8710097609996974 _WARN_ = >> Second observation... EM_CLASSIFICATION = BAD EM_EVENTPROBABILITY = 0.41037331440653074 EM_PROBABILITY = 0.5896266855934693 _WARN_ =

C H A P T E R

3

Notes and Tips

Missing Values In the SAS System, missing numeric values are represented by IEEE Not-aNumber values. An IEEE Not-a-Number, or NaN, is an IEEE floating-point bit pattern that represents something other than a valid numeric value. These numbers are not computationally derivable, which means that the bit pattern will never be generated by an arithmetic operation. The exact bit pattern for the NaN is different on different operating systems. For C scoring, the NaN bit pattern must be set in the cscore.h header file. Text in the body of cscore.h contains recommendations for NaN values that you should use with systems that are supported by SAS. NaNs can be problematic if they are not handled correctly. When you develop a scoring system, you should consider how to handle numeric data that contains missing values. In the C language, it may be necessary to use a C function to perform a bit check on each numeric value before you perform an operation on the numeric value. Examples of C functions that perform bit checks are available in cscore.h, as the NMISS function, and in the C main program csbasic.c used in this example as the AMISS function.


A P P E N D I X

1

Programming Information

General Code Limitations ......................................................................................................... 25 Supported Functions ................................................................................................................. 26 Supported SAS Operators ......................................................................................................... 27 Arithmetic Operators ......................................................................................................... 27 Comparison Operators ....................................................................................................... 27 Logical Operators ............................................................................................................... 27 Other Operators ................................................................................................................. 27 Conditional Statement Syntax .................................................................................................. 28 Variable Name Length .............................................................................................................. 28 Character Data Length.............................................................................................................. 28 Extended Character Sets ........................................................................................................... 28

General Code Limitations The SAS DATA step language is a flexible and powerful development environment. The Enterprise Miner component that generates C and Java scoring code supports only a small portion of the syntax, expressions, and functions that the SAS System supports. Every effort has been made to ensure that the DATA step code that Enterprise Miner produces is compatible with the restrictions imposed by the C and Java code generation process. It is possible to create code in Enterprise Miner that cannot be correctly translated into C or Java code. This is particularly a problem with data transformations performed within Enterprise Miner. When you use the Enterprise Miner Expression Builder to create transformations, and you want to migrate your scoring code to C or Java, you must take great care to ensure that your data transformations are expressed using code structures that resemble C or Java structures as much as possible, in order to facilitate the correct generation of score code. It is best to attempt to structure DATA step code for any transformation to make it as much like C as possible. In other words, any SAS operand or function that is not native to the C or Java languages should be avoided in your data transformation expressions unless the operand or function is explicitly supported by the C and Java code generation process.


Supported Functions The following SAS System functions are supported either directly by the target language libraries or by code that is distributed with Enterprise Miner: ARCOS(n); ARSIN(n); ATAN(n); CEIL(n); COS(n); COSH(n); c1= DMNORMCP(c1,n1,c2); c1 = DMNORMIP(c1,n1); n2 = DMRAN(n1); Similar to the SAS system function RANUNI n2 = EXP(n1); n2 = FLOOR(n1); INDEX(c1, c2); INT(n1); c1= LEFT(c1); n1 = LENGTH(c1); n2 = LOG(n1); n2 = LOG10(n1); nx = MAX(n1, n2, n3, …); nx = MIN(n1, n2, n3, …); n1 = MISSING(); nx = N(n1, n2, n3, …); nx = NMISS(n1, n2, n3, …); n2 = PROBNORM(n1); PUT((,fmtw.d); n2 = SIN(n); n2 = SINH(n); n2 = SQRT(n); c2 = STRIP(c1); c2 = SUBSTR(c1, p, n1); /* n is not optional */ SUBSTR(c1,p,n1) = strx; n2 = TAN(n1); n2 = TANH(n1); c1 = TRIM(c1); c1 = UPCASE(c1); Note: n1, n2, …, nx indicates numeric variables, and c1, c2, …, cx indicates character variables.

Appendix 1 27

Supported SAS Operators Arithmetic Operators SAS System Symbol

C *Score Equivalent

+ * ** /

+ *

Definition addition subtraction multiplication exponentiation division

pow();

/

Comparison Operators SAS System Symbol Mnemonic EQ = NE ^= GT > LT < GE >= LE < >= > / tstest1/cclist.txt 2>&1

-V

Causes subprocesses to print version information to stderr

-v

Enables verbose mode

-b

Creates a shared library rather than an executable file. The object files must have been created with the +z or +Z option to generate position-independent code (PIC).

+Z

Generates shared library object code with a large data linkage table (long-form PIC). +DD64 Generates 64-bit object code for PA2.0 architecture.

+DD64

Generates 64-bit object code for PA2.0 architecture

-q

Causes the output file from the linker to be marked as demand loadable. For details and system defaults, see the ld(1) description in the HP-UX Reference Manual

-Idir

Add the directory dir to the head of the list of directories to be searched for include files by the preprocessor

-o filename

Places the output in file filename

/tstest1/Score.c

The absolute name of the SAS Enterprise Miner generated C scoring source file

/sas920/safmts/jazxfbrg

is the Standalone formats link library

48 Enterprise Miner C and Java Score Code Basics

H6I- HP/UX on Itanium OS Name

HP/UX 11.23

Version

HP-UX B.11.23 U ia64

Compiler

HP aC++/ANSI C B3910B A.06.06 [Nov 7 2005]

Compiler Documentation

http://docs.hp.com/en/5921/c_index.html

Compile command

/usr/bin/cc -v -V -b +Z +DD64 -q -I/ tstest1/headers -o/ tstest1/libtstest1 / tstest1/Score.c /sas920/safmts/jazxfbrg >> / tstest1/cclist.txt 2>&1

-V

Causes subprocesses to print version information to stderr

-v

Enables verbose mode

-b

Creates a shared library rather than an executable file. The object files must have been created with the +z or +Z option to generate position-independent code (PIC).

+Z

Generates shared library object code with a large data linkage table (long-form PIC). +DD64 Generates 64-bit object code for PA2.0 architecture.

+DD64

Generates 64-bit object code for PA2.0 architecture

-q

Causes the output file from the linker to be marked as demand loadable. For details and system defaults, see the ld(1) description in the HP-UX Reference Manual

-Idir

Add the directory dir to the head of the list of directories to be searched for include files by the preprocessor

-o filename

Places the output in file filename

/tstest1/Score.c




Appendix 6 49

R64 – AIX on Power OS Name

AIX

Version

AIX Version 3.5

Compiler

IBM XL C Enterprise Edition for AIX, Version 7.0.0.4


http://publib.boulder.ibm.com/infocenter/c omphelp/v8v101/index.jsp

Compile command

c99 -q64 -G -qlibansi -qarch=com -I/tstest1/headers /tstest1/ Score.c /sas920/safmts/jazxfbrg -lm -o /tstest1/libtstest1

-qarch=com

produces object code that will run on all the 64-bit PowerPC(R) hardware platforms but not 32-bit-only platforms

-q64

Generates 64-bit code

-qlibansi

Configures the optimizer to generate better code because it will know about the behavior of a standard function

-G

Specifies the linker is to create a shared object enabled for runtime linking

-Im

Specifies the standard math library for linking. Some configurations may not require this.

-I dir

Specifies an additional search path for #include filenames

-o filename

Specifies an output location and name for the shared library

/tstest1/Score.c





S64 - Solaris on SPARC OS Name

Solaris 9

Version

SunOS 5.8 Generic February 2000 (a.k.a. Solaris 8)

Compiler

Sun C 5.7 Patch 117836-02 2005/03/23


http://docs.sun.com/app/docs/doc/819-3688

Compile command

cc -v -G -xtarget=ultra3 -xarch=v9a -xcode=pic32 -I/tstest1/headers -o /tstest1/libtstest1/tstest1/ Score.c/sas920/safmts/jazxfbrg

-G


-xcode=pic32

Generates position-independent code for use in shared libraries (large models). Equivalent to – KPIC.

-xtarget=ultra3

Specifies the target system for instruction set and optimization

-xarch=v9a

Specifies the instruction set architecture (ISA). If you use this option with optimization, the appropriate choice can provide good performance of the executable on the specified architecture. An inappropriate choice results in a binary program that is not executable on the intended target platform.

-I/tstest1/headers

Adds the specified directory to the list of directories that are searched for #include files

-o/tstest1/libtstest1

Specifies the output file filename instead of using the default filename of a.out. The specified filename cannot be the same as the source file. This option and its arguments are passed to ld(1).

/tstest1/Score.c




Appendix 6 51

SAX – Solaris 10 x64 (x64-86) OS Name

SunOS

Version

SunOS 5.10 i86pc

Compiler

C 5.9 SunOS_i386 Patch 124868-01 2007/07/12


http://docs.sun.com/app/docs/doc/819-3688

Compile command

Generic January 2005

cc -V -v -G -xtarget=opteron -xarch=amd64a -KPIC -I/tstest1/headers -o /tstest1/libtstest1/tstest1/ Score.c/sas920/safmts/jazxfbrg

-G


-xtarget=opteron

Specifies the target system for instruction set and optimization

-xarch=amd64a

Specifies the instruction set architecture (ISA). If you use this option with optimization, the appropriate choice can provide good performance of the executable on the specified architecture. An inappropriate choice results in a binary program that is not executable on the intended target platform.

-KPIC

generates position-independent code for use in shared libraries

-I/tstest1/headers

Adds the specified directory to the list of directories that are searched for #include files

-o/tstest1/libtstest1

Specifies the output file filename instead of using the default filename of a.out. The specified filename cannot be the same as the source file. This option and its arguments are passed to ld(1).

/tstest1/Score.c





Your Turn We welcome your feedback. •

If you have comments about this book, please send them to [email protected]. Include the full title and page numbers (if applicable).

•

If you have comments about the software, please send them to [email protected].

SAS Publishing Delivers! ®

Whether you are new to the work force or an experienced professional, you need to distinguish yourself in this rapidly changing and competitive job market. SAS Publishing provides you with a wide range of resources to help you set yourself apart. Visit us online at support.sas.com/bookstore. ®

SAS Press ®

Need to learn the basics? Struggling with a programming problem? You’ll find the expert answers that you need in example-rich books from SAS Press. Written by experienced SAS professionals from around the world, SAS Press books deliver real-world insights on a broad range of topics for all skill levels.

SAS Documentation

support.sas.com/saspress

®

To successfully implement applications using SAS software, companies in every industry and on every continent all turn to the one source for accurate, timely, and reliable information: SAS documentation. We currently produce the following types of reference documentation to improve your work experience: • Online help that is built into the software. • Tutorials that are integrated into the product. • Reference documentation delivered in HTML and PDF – free on the Web. • Hard-copy books.

support.sas.com/publishing

SAS Publishing News ®

Subscribe to SAS Publishing News to receive up-to-date information about all new SAS titles, author podcasts, and new Web site features via e-mail. Complete instructions on how to subscribe, as well as access to past issues, are available at our Web site.

support.sas.com/spn

SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Other brand and product names are trademarks of their respective companies. © 2009 SAS Institute Inc. All rights reserved. 518177_1US.0109