bpo management system Essay

1. Introduction 1.1 Purpose This document describes the high level design for the CHART/EORS Intranet Mapping Application and the CHART Internet Mapping Application. The purpose of this design is to show the high-level technical approach to meeting the requirements defined in system requirements specification. This serves to identify the architecture of the system and high-level interactions between major system components. 1.2 Objectives Identify and describe the software architecture for the system. Provide  high-level approaches to various technical challenges. Provide a guide for future development efforts, such as detailed design and coding. 1.3 Scope This high level design encompasses the approach for meeting the requirements as defined in the documents CHART/EORS Intranet Mapping System Requirement Specification and CHART Internet Mapping System Requirement Specification. 2. System Architecture 2.1 Overview The following diagram shows the system architecture used by CHART mapping applications.  The system design utilizes web based multi-tier system architecture. The data storage is managed at the data tier by the databases. The main business logics are hosted in the two applications in the web server. Because mapping is an area that there are many requirements related to client side interactions with the graphic content of the application, application logics are partitioned based on the most appropriate location to execute them. Some are located on the client browsers to provide instantaneous feedback to the user. The general system operation flow involves the following: 1. Data updates from various sources such as the CHART II CORBA events, EORS data inputs, device and event editing modules are stored in the databases. 2. When a mapping application receives a mapping request, it sends the image map generation request to the ArcIMS map server. The ArcIMS server retrieves the map data from the databases and creates a rendering of the map and saves it as a raster image file. The mapping application generates HTML pages embedded with the image and sends it to the browser client. 3. For the CHART Intranet mapping client, the application also generates the dynamic content in VML format, which encodes the device and event information in vector format. This enables the application to update the dynamic data without having to reload the whole map image. This avoids the heavy load on the map server when the application scales up. 4. When the images and VML  data arrives at the client browser, the client displays the map to the user. The user can interact with this data on the client. 2.2 SQL Server and ArcSDE ArcSDE from ESRI allows managing of geographic information in commercial databases such as SQL Server, Oracle, DB2 and Informix. ArcSDE provides functionalities to efficiently store and retrieve spatial information using spatial indexing mechanisms. ArcSDE provides a set of API and administrative utilities that help manage the spatial data storage. For the CHART mapping systems, the combination of ArcSDE and SQL Server manages the spatial information in the relational database. ArcSDE adds spatial functionalities without disrupting standard SQL database capabilities. 2.3 Map Server (ArcIMS) ArcIMS includes a few components that will play important roles in the CHART mapping application. The workhorse component that processes the data and generates maps is the spatial server. Managing the spatial servers is the ArcIMS Application Server, which monitors each spatial server’s activity and brokers map requests to the least busy spatial server. The detailed interaction of a map request is as the following: When the ASP.Net page receives a map request, it parses it and uses the ActiveX Connector object model to construct a map request. The connector then sends the map request in ArcXML format to the Application Server. The Application Server then finds the least busy spatial server and forwards the map request to it. The spatial server performs the query against the ArcSDE database, retrieves the data and renders them into a raster image file. The location of the file is then sent back to the connector and the ASP.Net page writes it back to the client as HTML page with the image embedded in it. 2.4 Web Server (IIS) .Net Framework and ASP.Net The web server hosts and publishes content to the client browser. In the case of the CHART mapping applications, most of the content is dynamic content generated by ASP.Net modules. When IIS recognizes a page being an ASP.Net module (an aspx extension), it passes the request to the .Net Framework to load the module and handle the request. The ASP.Net pages are then loaded into memory and executed. The .Net Framework provides many utilities such as garbage collection, tracing, just-in-time compilation that manages the execution of ASP.Net modules. The ASP.Net page modules are where the CHART mapping application logic is coded. The web server also provides security via the Secured Socket Layer (SSL), allowing interactions between the user’s browser and the web server to be encrypted when necessary. 2.5 Security 2.5.1 Network Level Security Network layer security will be managed by the network security configurations like firewall and RSA secure ID. 2.5.2 Secured Socket Layer (SSL) MDOT has a certificate server to provide digital certificates for the SSL configuration. The server name must remain consistent with the certificate. All links shall use the same server name, otherwise, if the server is referred using an IP address or a local server name, etc., the user will see an alert indicating the certificate is in-consistent with the resource. IIS supports the configuration of one folder in the web application requiring SSL while other portion does not. The session information remains consistent between SSL portion of the web site and the non-SSL portion. 2.5.3 Enterprise User Enters Read-Only View Many of the CHART mapping functionalities are for display and reviewing data, i.e. a read-only view. The design allows enterprise viewers and CHART users to access the read-only portion of the web site without having to input user name and password. This also enables CHART users to reach the viewing area without having to enter their login information again. When system receives a user request to enter the secured area, the system checks whether the current session has been authenticated. If not, system displays login screen. The user shall enter their CHART user name and password. Upon receipt of the user name and password, the system checks it  against the CHART II database’s user tables. If they are authenticated, the system stores the user information in the session. The session will be managed in the server until the configured timeout expires. All subsequent requests from the same user session will inherit the same authorization information for the user. 2.5.4 CHART User Enters Editing Area Other applications, like future versions of CHART II and CHART Lite, can launch the map editing URL via the HTTPS protocol. The user name and password can be sent via https request. The system verifies their authentication information against the CHART II user database using an OLEDB/ODBC connection. If the authentication information is correct, the system will store this information in the session. The user will be redirected to the map page. If the authentication is rejected, the user request will be redirected to the login screen to reenter the authentication information. Associating a CHART user with an op-center/default map view area: Based on CHART II R1B3 database design, users are not associated with an op-center; rather, the user specifies an op-center during logon. In order to display a default map view area based on an op-center, an external application launching the CHART mapping application will also need to pass in the operation center name to initialize the map to the associated extent. Passing user name and password in URL request: The mapping site shall have a module that verifies the user name and password, then forward the page to the map page, hence avoiding showing the password on URL address box. At the current time, without the full integration with CHART II and CHART Lite, the system will expect plain text user name and password. In the future, an encryption/decryption algorithms agreed between the systems can be added to achieve higher security. 2.5.5 EORS Security Currently, the EORS security has not been implemented. EORS functions will be hard-coded with security configuration. 3. Network Configuration The design above depicts CHART network configuration as the Internal network, a Demilitarization Zone (DMZ) network for hosting the web server and connecting out to the external Internet network. There will be two firewalls, one between the Internet and the DMZ network and another one between the DMZ network and the internal network. The map server and database servers are to be hosted in the internal network for maximum security. The initial configuration calls for two physical computers to host the map servers and database servers. In the future, if the system needs to scale up, additional physical servers can be added. The Intranet web server can optionally be hosted on the load-balanced virtual server too. 3.1.1 Map Server Load Balancing The design achieves load balancing by a combination of Windows 2000 Advanced Server Network Load Balancing (NLB) Service and the ArcIMS Application Server. The system utilizes two physical server computers. The two servers are configured with NLB. NLB works on the TCP/IP level. Any incoming traffic from web server to the virtual server IP address is load balanced between the two application servers by NLB. ArcIMS Application Server operates at the application level, monitoring each spatial server’s load and operation. When a spatial server is busy, it directs the map request to idling spatial server(s). Each physical map server hosts one ArcIMS Application Server and two ArcIMS Spatial Server instances. An application server failure forces NLB to direct new connections to the remaining application server. When the failed server is recovered, new client connections should once again be shared between the two servers. The two spatial server instances are â€Å"cross registered† to the application servers. As shown in the diagram, Spatial Server A1 and A2 are registered to Application Server B and A correspondingly. This arrangement ensures that when a spatial server is down, the application server can still  utilize the spatial server from the other server to serve the map request and the application server continue to function. Also, this configuration also allows ArcIMS to load balance at the Spatial Server level as opposed to just the network traffic level, which is what NLB provides. This configuration can withstand an Application Server failure, a Spatial Server failure, a simultaneous Application/Spatial Server failure or hardware failure of one of the physical map servers. Using two map servers with network load balancing should provide high-availability load balanced ArcIMS web site. 3.1.2 Database Load Balancing By running two SQL Server and ArcSDE instances with NLB to balance the load, the system can achieve high availability at the database server layer. The database servers are completely independent and share no hardware components. This type of availability is achievable with the standard edition of SQL Server. The two database servers are setup with Transactional replication. One of the two SQL Servers is configured as the publisher and the other one as a subscriber. All the data modification such as insert, delete and update will be performed on the publisher and changes are replicated to the subscriber. Transactional replication can provide very low latency to Subscribers. Subscribers receiving data using a push subscription usually receive changes from the Publisher within one minute or sooner, provided that the network link and adequate processing resources are available (latency of a few seconds can often be achieved). When the web server and map server requests use the virtual IP address on the load-balanced group of database servers, they are directed to the database server with the least amount of load. If one of the database servers goes down due to hardware failure, NLB detects that this server is down and no longer directs database requests to this machine. The remaining machine handles the database requests and apart from a slight drop in performance the users are unaware that a database server has failed. When the hardware is fixed the offending machine can be brought back online. One limitation exists for this design. It happens when the publisher database is down. In this situation the data updates cannot be committed until the publisher database comes back. But at the same time, all read access from the Internet and Intranet server could still be directed to the secondary server. In the case when the publisher data is going to be down for extended time period, system configuration need to allow system administrator to change the configuration so that the replica will serve as the main database. Compared with clustering solution, this system design provides the maximum database availability and performance benefit. The databases that need to be replicated would include: 1. Background map database. Background map data does not change often. A snapshot replication is sufficient for replicating data updates in one database to the other. 2. CHART/EORS Spatial Database CHART/EORS spatial database stores CHART and EORS device and event information with spatial data. They are dynamically updated throughout the day. Transactional replication will be setup to ensure that data change in one database gets replicated to the other one. 3. SDE metadata. In general, the system can continue to provide access of map and data to both the Internet and Intranet users in the case of failure of any one component in the system. The only exception is that when the publisher database is down, the new data cannot be updated into the system. Users will get delayed information. 3.1.3 DMZ Configuration CHART is currently implementing a Demilitarization Zone (DMZ) network to enhance the network security. This entails creating a separate network for the web server computer(s) and separating it from the internal network with a firewall. In an ideal world, the DMZ would have no physical connection to the internal network. This would require two separate map server setups to serve the  Internet and Intranet users. The recommended way to implement is to disallow any access from the DMZ to the internal network, but allow access from the internal network to the DMZ. In other words, allow out-bound connections. On each of the ArcIMS server computers, mount a network drive to a shared drive on the Web server. Each ArcIMS spatial server would write the output raster image files to the location on the web server to be delivered to the Internet client browsers. 4. Database Organization To reduce the dependency and operation interference between the spatial data and the attribute data, the EORS spatial database and CHART spatial database will be created as two SQL Server databases. To reduce the performance overhead when joining data between the spatial and attribute data, the EORS spatial database will reside on the same database server(s) as the EORS database. 5. Technical Challenges 5.1 Map Display Mechanism CHART Intranet mapping application requires that changes in event and device data be reflected on all map clients in a near-real-time fashion (within 5 seconds). To do so via the traditional raster map publishing mechanism will result in all clients retrieving updated map every 5 seconds or at least when event/device status update requires a new map to be generated. When there is large number of users of the system, it will result in a high map server load in a concentrated short time period. To resolve this issue, the project team reviewed various technical approaches and summarizes their advantages and disadvantages as the following: 5.1.1 Raster (JPEG, GIF or PNG) Image This is a popular approach that utilizes the basic image display functionality of web browsers. It utilizes the server processing power efficiently. The disadvantages are that the images have limited client side intelligence, leaving most of the computation concentrated on the server. It’s capability of handling large number of concurrent map requests is limited. Generally, one map server can support 4-8 requests per second. For  CHART’s situation, when an event changes status, if a new map image needs to be generated, it would be about 40 requests per second (200 users at 5 second update interval). Many servers will be required to support the load. With the license fee involved with using GIF format, we will not use GIF for map publishing. Compared with JPG format, PNG graphics do not have the â€Å"bleeding† effect inherent with the JPEG compression algorithm. With the map displaying lines rather than continuous tone images, it is much cleaner. PNG also results in a smaller file, which translates into faster download times for client. The only JPG advantage is server side image generation times. It is recommended to utilize PNG for the Intranet application to produce highest quality images for standardized IE browser while utilizing JPG for the Internet to allow for support of as many browsers as possible. Also, the reduction in image processing time should deliver better web image generation performance. 5.1.2 XML Based Vector Graphics 5.1.2.1 Vector Markup Language (VML) VML is a XML based W3C standard in describing vector graphics. Basically, it encodes the vector coordinates of points, lines and polygons in XML format. The support of VML is included in Internet Explorer 5.0 and later. There is no download needed to display VML encoded vector graphics. It also has built-in support for style sheet and scripting. This makes it possible to modify the display properties and positions of the vector graphics using the JavaScript on the client-side browser. Using this functionality, we can dynamically update the display of devices and events. 5.1.2.2 Scalable Vector Graphics (SVG) Scalable Vector Graphics is another XML-based W3C standard format for vector graphics. Compared with VML, it requires Java or ActiveX based plug-in to be displayed. Also, based on review of the plug-ins (SVG Viewer by Adobe), there is not as much support for scripting as for VML. 5.1.3 ArcIMS Java Viewer ArcIMS includes a Java Viewer, which provides a Java Applet that can be customized to a certain extent to display vector encoded GIS data on the client side. It requires a download to the client. The Java Viewer reads vector data from ArcIMS feature server encoded using an ESRI proprietary compression format, which makes it difficult to implement special features such as WSMS offsetted road networks because they need to be offset dynamically based on map scale. 5.1.4 MapObjects Java MapObjects Java from ESRI provides a set of Java-based objects for GIS functionalities. It has an extensive set of functionality that can satisfy the requirements. But, it requires a license fee of $100/seat, or comparable server-based licensing. It also requires a download to client machine to run it. 5.1.5 Summary Based on the research above, the project team recommends the following design: Use raster map for background map display (background data with SHA grid map are often large amount of data, suitable for server side processing) Use XML data format to transport the device and event data from server to browser client. Use JavaScript to create and update the VML vector data elements to display the dynamic layers including all the CHART/EORS devices and events. The diagram above illustrates the map display mechanism: 1. Map server reads the spatial data from background database and sends the published map image to the client browser to be displayed as background. 2. Device and event information is broadcasted from the CHART II system in the form of CORBA events. 3. CORBA event listener receives the event broadcast and saves the data into CHART spatial database. 4. CHART/EORS device and event data are published in XML format to the client. Client browser parses the XML into a XML Document Object Model (DOM) using the XML parser. 5. The client browser then iterate through the DOM tree structure and create corresponding VML elements based on the XML device and event data. The VML elements are displayed on the top of the background raster map image. 6. At a pre-configured interval, the browser client retrieves update of device and event data from the IIS server in XML format and update the VML display  based on the updated information. 5.2 Automated Refresh of Device and Event Data CHART/EORS device and event information needs to be updated at a pre-configured interval. They should be updated separate from the background map to reduce the load on the map server. The technical approach to achieve this will be to use a hidden frame to send the request to retrieve updated device and event data and receive the response. The response will package the data in XML file to be parsed into a document object model (DOM) and display the data on map. The request can be to retrieve all data or only retrieve data newer than last retrieval. When the new device/event is received and it requires changing the display of the device/events, the style assignment for the elements can be changed to update the device and event display.   The VML elements will be sent using real-world coordinates (Maryland State Plane 1983). After the data has been retrieved to the client side, the VML map layers can be dynamically projected using the â€Å"local coordinate space†. When user zooms or pan the map, the VML will be projected using the updated coordinates to fit the new map extent without going back to the server to retrieve new data set. 5.3 Inter-Frame Client Script Synchronization The map page has a few frames and the browser loads them asynchronously. Scripts in one frame may call scripts in another frame that may not have been loaded. The approach to resolve this is to add client-side exception handling and verification routine to ensure that the script is called always after the frame is loaded. 5.4 Assigning and Editing Event Location The dynamic nature of VML elements in the browser allows adding and modifying VML elements by scripting. When the user clicks or drags the mouse on the map, client-side script manages the transformation of screen coordinates and real-world map coordinates. The coordinates are sent back to the server’s secured URL where the information is extracted and saved to the database. 5.5 Scalability The CHART mapping application serves not only the Intranet users, but also Internet browser clients. During emergency situations, the load on both the Internet and Intranet servers could get extremely high. The system must be able to scale up to serve large amount of users. The technical approach to solve this issue involves two main facets. As described in the network configuration section, the system will employ network load balancing and allows adding additional hardware in the future. The system should also be able to utilize the caching feature of IIS and ASP.Net to scale up without significantly increase hardware investment. ASP.Net allows caching configuration for individual page modules, such as whether the page is cached and how long it is cached. After the application is deployed, these caching settings can be configured on the web pages. For example, if it is determined that the Internet mapping can be up to 3 seconds late, by setting caching time to 3 seconds, all requests from the Internet will receive a cached response without creating additional load on the map and database server. 5.6 Exception Management and Recovery CHART II keeps its clients updated via a push model using the CORBA Event Service. The Event Service does not guarantee delivery; therefore it is possible for event data to be lost/dropped (although in practice, this is rare). To account for this possibility, the CHART Web Event Listener will refresh its information about the status of devices and traffic events from CHART II at a configurable interval. Also, each time the Event Listener is started, it will retrieve all relevant data from CHART II. Thus, the update model becomes a push model with an occasional pull to be safe. This process will be used to recover from the following situations: 1. The Event Listener was down and did not receive new data from CHART II. 2. CHART II CORBA event(s) occasionally dropped while the Event Listener is up and running. Another likely scenario is that the CHART II server or service(s) restart. After a typical restart, the CORBA Event Service CORBA objects will be  recreated with the same characteristics allowing the Event Listener to continue to automatically receive CHART II CORBA events. As the CHART II services will not be processing events during this time, no events are likely to be missed. Therefore, the Event Listener does not need to do anything special to handle a CHART II server or service(s) restart. Sometimes CHART II maintenance will require that new (and different) Event Service CORBA objects be created. This might happen during a CHART II upgrade, for example. In this case, the Event Listener will need to be restarted so that it can pick up the new objects. Since this type of maintenance does not occur often and the Event Listener restarting is fast, the restart can be handled as part of the CHART II upgrade procedures. 5.7 Integration with ASP Code in EORS and CHART Web Application The CHART Intranet Mapping, replacing the existing EORS mapping application, will still be launched as a separate window by a URL string with a few parameters identifying the district, view type, etc. The impact on EORS web application should be limited to modifying the URL links. The current CHART Internet Mapping site uses â€Å"include† statement to include site navigation pages from upper level CHART web site’s pages. When upgrading Internet Mapping to ASP.Net, â€Å"include† statement is no longer used. Instead, a ServerXMLHTTP request can be formulated to request the text from the included ASP page and merge them into the mapping ASP.Net pages. The limitation of this implementation would be that the ASP.Net application couldn’t share the session and application variables from the ASP application. Currently, there are only a couple of them, such as database connection string. The ASP.Net mapping application will maintain a separate set of application variables. 6. User Interface Design 6.1 Intranet Map Site User Interface Design Here following is a high-level frame structure for the Intranet mapping site: 1. AppFrame is the highest-level frame that includes all the child frames. On the top of the page, there will be the title frame, which will host the  CHART icon. Also inside the title frame will be a group of tabs, such as Traffic, Roadway Weather, Message Sign, etc. 2. ToolsFrame hosts the map navigation and other map related tools. The ToolsFrame will also host menu system that allows the user to bring up data and other detailed information. 3. HiddenFrame will be used to submit and receive information from the server. 4. ContentFrame is further divided to a map frame on the left and a data frame on the right. The user shall be able rearrange the frame boundary to give more space to the map or data area. Data frame will display data as well as legend, layer control and other items when needed. 5. PromptFrame will display the current tool selected and instructions for user activities. Here is a screen shot of the preliminary user interface design: 6.2 Internet Map Site User Interface The overall CHART Internet mapping web site design will stay the same as current web site. The site will stay as part of the overall CHART web site by including the CHART navigation menus into the site. The site will not be using frames; instead, all elements will be laid out as HTML tables. 7. References 1. CHART/EORS Intranet Mapping System Requirement Specification 2. CHART Internet Mapping System Requirement Specification 3. Security and ArcIMS – ESRI White Paper 4. ArcSDE Configuration and Tuning Guide for Microsoft SQL Server – ESRI White Paper 5. ArcIMS 4.0 High-Availability Configuration Testing Using Network Load Balancing –ESRI White Paper 6. Vector Markup Language (VML) Specification – W3C 8. Terms and Glossary ArcXML – ESRI’s map request/response specification in XML format CORBA – Common Object Request Broker Architecture CSS – Cascading Style Sheets DOM – Document Object Model ESRI – Environment System Research Institute GIS – Geographic Information System GML – Geography Markup Language NLB – Network Load Balancing SSL – Secure Socket Layer SVG – Scalable Vector Graphics VML – Vector Markup Language XML – Extensible Markup Language