Remote access to PVSS projects and security issues DCS
computing related issues Peter Chochula
Slide 2
P. Chochula DCS Workshop, CERN, September 2004 2 Computing and
Network Infrastructure for Controls (CNIC) a new CERN-wide working
group dealing with security and system management issues has been
created ALICE nominated a contact person who will follow the CNIC
progress In later phase, the CNIC membership will be expanded to
include the domain managers
Slide 3
P. Chochula DCS Workshop, CERN, September 2004 3 Main points
addressed by CNIC Phase I Tools for system maintenance (NICEFC,
LINUXFC) Tools for setting up and maintaining different controls
domains Rules and policies for domains Rules for inter-domain
communication Rules for communication between domains and CERN
network
Slide 4
P. Chochula DCS Workshop, CERN, September 2004 4 CNIC decisions
are important for ALICE DCS We will provide feedback and advertise
our experience we will follow the recommendation and adopt
available tools ALICE pre-installation is starting First
deliverables of CNIC are expected for the end of 2005 For the
pre-installation ALICE DCS must provide own tools and define own
policies, these might be later modified according to CNIC
outcome
Slide 5
P. Chochula DCS Workshop, CERN, September 2004 5 Main tasks for
maintaining the ALICE DCS domain Operating system and software
deployment Software maintenance (upgrades and patches) Users
management Computer security Remote access to systems Computer
Monitoring
Slide 6
P. Chochula DCS Workshop, CERN, September 2004 6 Monitoring of
DCS computers A CERN-wide effort to provide common computer
supervision tools has been started LASER seems to be a system,
which will be deployed in the future ALICE DCS follows the
developments and is ready to evaluate the software once it becomes
available Simple version of computer monitoring tools based on
Framework PCMON will be used in the starting phase of
pre-installation
Slide 7
P. Chochula DCS Workshop, CERN, September 2004 7 FW PCmon tools
LINUX WXP Dead PCMON connection
Slide 8
P. Chochula DCS Workshop, CERN, September 2004 8 FW PCmon
tools
Slide 9
P. Chochula DCS Workshop, CERN, September 2004 9 Operating
systems and standard software deployment Computer infrastructure
for pre-installation will be mostly based on Windows Windows Server
2003 will be our server platform ALICE DCS will operate on own
Windows domain (decoupled from NICE)
Slide 10
P. Chochula DCS Workshop, CERN, September 2004 10 Software
installation ALICE DCS purchased licenses for Windows Server 2003
RIS Remote Installation Service is presently evaluated Remote boot
using the PXE protocol built-in on recent computers boot floppy
available for 32 mostly used network cards if needed Pre-configured
template operating system will be loaded from the server at
installation time Standard software will be deployed automatically
after OS installation
Slide 11
P. Chochula DCS Workshop, CERN, September 2004 11 Need for
standardization of software versions and target paths (see talk
related to conventions, presented at this workshop) RIS assumes a
standard destination for software, relicts on target drive will be
wiped out) Automatic installation of PVSS and standard software
will follow after OS deployment
Slide 12
P. Chochula DCS Workshop, CERN, September 2004 12 Software
maintenance Deployment of patches is a complicated task Possible
interference with installed software must be understood in advance
Installation of patches must not harm the controls system As a
consequence the NICE procedures are not applicable for DCS ALICE
DCS computer will use own patch deployment policies based on
SUS
Slide 13
P. Chochula DCS Workshop, CERN, September 2004 13 SUS Software
Update Services Standard software update services use the following
scenario: Update server publishes a list of available updates
Workstations connect periodically to this server and download the
software ALICE DCS SUS server downloads all patches from Microsoft
Only patches approved by administrator are published (become
visible) for workstations in the DCS domain
Slide 14
P. Chochula DCS Workshop, CERN, September 2004 14 ALICE SUS
server (pcaitdc14) Approved patch
Slide 15
P. Chochula DCS Workshop, CERN, September 2004 15 How to
assure, that the patches have been correctly deployed? If for some
reason patch deployment fails, this must be detected and corrected
Need for remote scanning Microsoft baseline security analyzer has
been evaluated MBSA connects to SUS server and receives a list of
approved patches MBSA scans workstations and checks the presence of
required software In addition, MBSA scans for system
vulnerabilities (like opened ports, dangerous unprotected shares,
missing policies etc.)
Slide 16
P. Chochula DCS Workshop, CERN, September 2004 16 MBSA for
ALICE DCS Computer to be scanned Scanning of a range is also
possible SUS Server
Slide 17
P. Chochula DCS Workshop, CERN, September 2004 17 MBSA security
report
Slide 18
P. Chochula DCS Workshop, CERN, September 2004 18 MBSA problem
tracking
Slide 19
P. Chochula DCS Workshop, CERN, September 2004 19 Additional
security related topics MBSA checks for existing vulnerabilities.
In fact a hacker could do the same Intrusion detection is also
needed First level of protection is provided by CERN security team
Second level - network attack countermeasures will be applied for
ALICE DCS SNORT intrusion detection system is being evaluated
Hardware protection (firewall), etc. are being studied Antivirus
protection will be needed (interference with running systems is
studied Deployment of users software will be possible only via
central DCS server
Slide 20
P. Chochula DCS Workshop, CERN, September 2004 20 Remote access
to controls system Windows Terminal server has been evaluated as an
option Clients exist (and were tested) for different platforms
including All flavours of Windows Linux MAC OS Remote user will
establish a secure RDP connection to Terminal Server and from here
he will connect to the DCS system using PVSS Remote UI
Slide 21
P. Chochula DCS Workshop, CERN, September 2004 21 Windows
Server 2003 Example: Remote access to temperature monitoring system
Simplified panel for remote monitoring DCS Device Access Complex
panel for DCS operator Secure (ssh) connection between PVSS
managers Secure (RDP) connection Firewall
Slide 22
P. Chochula DCS Workshop, CERN, September 2004 22 Evaluation of
the Terminal Server technology Extensive tests have been performed
this summer Naomi van der Kolk DCS summer student Influence of
remote UI on performance of the Terminal Server and Master DCS
project has been studied Involved computers were monitor via
modified Framework PCMON tool
Slide 23
P. Chochula DCS Workshop, CERN, September 2004 23 Terminal
server test setup Powerful computer (2x Xeon 2.8 GHz, 3 GB RAM)
running Windows 2003 Server has been used as a terminal server
Master project was running on P-IV 3 GHz equipped with 1 GB RAM A
number of P-IV computers were used as remote user stations In test
we studied impact of remote connections on CPU utilization and
memory allocation on Terminal Server, remote user station and
Master Project
Slide 24
P. Chochula DCS Workshop, CERN, September 2004 24 Windows
Server 2003 Windows Server 2003 Windows XP Pro Windows XP Pro
Windows XP Pro Computer Infrastructure for Remote Access Tests DCS
Private Net. 192.168.39.0 CERN Net. Terminal serverRouter PVSS
Master Project Remote User Windows XP Pro
Slide 25
P. Chochula DCS Workshop, CERN, September 2004 25 Performed
tests Master project generated 50000 datapoints and updated 3000 of
them each second Remote user displayed 50 or 200 of them Tests were
performed for Different number of remote sessions Different number
of datapoints updated on remote screen Connections were made to a
busy and idling master project
Slide 26
P. Chochula DCS Workshop, CERN, September 2004 26 Terminal
Server Load Master project stopped Master project running Remote
panel displays 50 values Remote panel displays 200 values Master
project generated 50000 datapoints and updated 3000 /s
Slide 27
P. Chochula DCS Workshop, CERN, September 2004 27 Load of the
workstation running the master project Remote client disconected
Remote client connected Master project stopped Master project
running
Slide 28
P. Chochula DCS Workshop, CERN, September 2004 28 Computer
loads for large number of remote clients Terminal Server #clients
Average CPU load [%] Mem [kB] 6011.2 2781282 5511.0 2788719 4513.8
2790181 3512.0 2672998 259.7 2067242 157.2 1448779 54.2 934763 04.9
666914 Workstation running the DCS project #clients Average CPU
load [%] Mem [kB] 6085.1579666 5586.6 579829 4584.9579690
3581.3579405 2580.9 579384 1581.4 579463 583.0580003 083.7579691
Master project generated 50000 datapoints and updated 3000 /s.
Remote client displayed 50 values at a time
Slide 29
P. Chochula DCS Workshop, CERN, September 2004 29 Conclusions
on TS tests No performance problems were observed for master
project TS load depends on number of external connections. CPU of
TS easily supported up to 60 external connections (no tests were
made beyond this number) Memory consumption was measured to be
~80MB per remote session. Memory paging did not cause observable
drop of remote display performance ALICE DCS will deploy TS in
pre-installation phase and communicate results to CNIC
Slide 30
P. Chochula DCS Workshop, CERN, September 2004 30 Standard
services provided by ALICE server(s) DCS domain controller RIS
patch deployment security scans and intrusion detection DIM name
service WEB service, Internet name service Remote access to PVSS
projects Tools for deployment of users software TFTPD for CAEN
firmware upgrades please express you needs
Slide 31
P. Chochula DCS Workshop, CERN, September 2004 31 This slide is
important All software needs to be carefully tested before its
deployment on DVS network Unlike the other online systems, the DCS
need to use a large number of third-party software Some components
(DIM servers, OPC servers) may badly interfere with software
patches A test setup in the DCS lab will be used to test the
software We need your software well in advance
Slide 32
P. Chochula DCS Workshop, CERN, September 2004 32 Conclusions
Computing infrastructure is getting ready for pre- installation
Windows Server 2003 will be our server platform Provided services
will cover RIS, software updates and security scans, DCS related
services and remote access Computer monitoring will be based on FW
PCMON, possibility for transition to other tools (e.g. Laser) will
be evaluated Quality of service depends on users feedback we cannot
prepare for secret requests
Slide 33
P. Chochula DCS Workshop, CERN, September 2004 33 Computing
related questions CO1: Which operating systems do you plan to use
in your DCS. Please specify also the OS versions and the hardware
platforms. Present standards are Windows XP SP1 and CERN RedHat
7.3.4 Longhorn and CEL3 will be the successors SP2 will be deployed
soon presently it is being tested with PVSS We need to know your
requirements, especially if you need to install some non-standard
systems (but this is strongly discouraged)
Slide 34
P. Chochula DCS Workshop, CERN, September 2004 34 Computing
related questions CO2: which applications besides the PVSS-II do
you need to run on your computers? (Please list all software
packages, also the obvious ones (like root, MS Office etc.)) The
list is essential for preparation of software deployment policies
The software must be tested with the rest of the DCS We need to
understand the dangers (e.g. even a harmless installation of C++
compiler could cause a serious security risk flaw in embedded SQL
server)
Slide 35
P. Chochula DCS Workshop, CERN, September 2004 35 Computing
related questions CO3: what are your requirements for remote access
to the DCS system (please give details on number of remotely
monitored parameters, frequency of their updates, and number of
simultaneous remote sessions). You input is needed for correct
infrastructure planning
Slide 36
P. Chochula DCS Workshop, CERN, September 2004 36 Computing
related questions CO4: What are the requirements for additional
data storage on DCS computers? CO5: Which remote systems (located
at CERN) do you need to access from the DCS network? (e.g LXPLUS
etc.) We need to define archiving (backup) policies Secure access
to remote systems has to be provided