© Integra 2014 / Page 1. © Integra 2014 / Page 2 IT Basic Troubleshooting What makes DocuTrack 5 work Don Drohman DocuTrack Support Manager, Client Services

© Integra 2014 / Page 1


IT Basic TroubleshootingWhat makes DocuTrack 5 work

Don DrohmanDocuTrack Support Manager, Client Services

Jan 22nd, 2014


IT Basic TroubleshootingWhat makes DocuTrack 5 work

This class will give you a basic overview and tools to understand the key components, technologies, file structures, Supporting applications and effective use of Integra Support resources, in order to provide technical support for DocuTrack in your Pharmacy.

This is an open discussion. Feel free to ask questions!This is an open discussion. Feel free to ask questions!


Architecture components overviewThe pieces that make DocuTrack work

Server components

The DocuTrack Application itself

• %PROGRAMFILES(x86)%\Integra\DocuTrack

• %PROGRAMFILES(x86)%\Integra\SupportTools\utils (Support Tools)

Microsoft .NET Framework 3.5.1 + 4.0 + all related critical and security Windows Updates

Operating system

• Microsoft Server 2008 R2 SP1

• Microsoft Server 2012 (Starting with DT 5.3.0)

• Microsoft Server 2013 R2 (Starting with DT 5.4.0)

Microsoft SQL Server

• SQL Server 2008 R2 SP2

• SQL Server 2012 SP1 (starting with DT 5.3.0)

Source: DocuTrack system requirements and build instructions


Architecture components overview (cont.)The pieces that make DocuTrack work

Server components (continued)

Windows Roles

• Application Server (COM objects, etc.)

• File Services (Folder sharing, storage management etc.)

• Web Server (IIS, SMTP)

Windows Features

• .NET 3.5.1 Features

• Remote Administration Tools (Role – IIS Web tools / Feature – SMTP admin)

• Windows Process Activation Service

• telnet client (optional)




Server components (Continued)

Local Users

• IntegraSVC – a service account.

• IntegraSupport – a local administrator account.

• Reports– user or group with access to SQL Reports

• Forms – user or group with access to the Forms folder

• QS1Delivery – user with access to the QS/1 delivery manifest data drop point.

• Local Administrator account – Pharmacy managed account




Client Components

Client Installer

• Installer currently provided as an .exe (.msi has been requested of PM)

• DT 5.3.x and above also installs MS Visual C++ 2008 SP1 Redistributable.

• Supported on WinXP, Vista, Win7 and Win8 (as of DT 5.3.3)

• Current client for your server will always be located at:

• http://<server name or IP>/DTSupport

• Requires Microsoft .NET Framework 3.5.1 + 4.0 + all critical Windows updates

• User needs sufficient rights to install an application.

• Where supported, use the RunAs Administrator

• Inbound Windows Firewall rules will be added to OS’s, where supported (not XP)

Auto-Update (AppLife)

• Disable AppLife Update Service in a managed / Citrix environment.

• When logging in, client will auto-detect a new version and initiate update.

Source: DocuTrack 5.3.3 Installation and Upgrade guide



Other Components (external to DocuTrack)

Fax System (more later)

IIS (minimal as of DT 5.4)

• Certificate management, current client web page

SMTP

• Fax delivery (FACSys)

• Scanned image delivery from Multi Function printers.

• Outbound alerts

• ZAN Printer delivery (file drop not reliable)

• **Not an Internet grade or facing SMTP mail server.

Interfaces to PIS, EMAR, or other systems

Barcode recognition software (ClearImage by Inlite Research)

• www.inliteresearch.com

Source:



Other Components (external to DocuTrack) (cont.)

Task Scheduler (for example)

• Backup jobs

• Pop3 mail downloader

Dell OpenManage Server Administrator (OMSA) (only on Dell Servers)

• Drive array monitoring and manipulation

• Hardware Alerts

Pop3 mail client, Image printer – more later in presentation

Source: DT Support Tools



Support tools (external to DocuTrack)

BareTail / BareGrep (www.baremetalsoftware.com)

• Log searching and review

.NET Webservices Studio (archive.msdn.microsoft.com/webservicestudio20)

• Troubleshoot Web Services

RDC / ScreenConnect – (www.screenconnect.com)

• Integra Remote support tool

LabTech (www.labtechsoftware.com)

• Integra Remote deployment and metrics gathering tool

Source: DT Support Tools



File System Folder Structure

Drive Partitions (Integra Provisioned hardware)

• C: OS (80G min. for Win2008R2) – Ideally Raid1

• D: DATA (Size based on growth , but 100G to start) - Ideally Raid5

• 1 Global hot spare drive

Import Location (the old \incomingmail folder)

• %Path.DataRoot%\Import

Pre-Import Location

• %Path.DataRoot%\Documents\System\PreImport

Document Store (the old \scans folder)

• %Path.DataRoot%\Documents\Files\<year>\<month>\<day>\<hour in UTC>

• %Path.DataRoot%\Documents\Files\Shadows\Preprocessed\

Source: DocuTrack internal training and system builds

Path.DataRoot is typically D:\DocuTrack , but is configurable.Path.DataRoot is typically D:\DocuTrack , but is configurable.



File System Folder Structure - ProblemScans (continued)

ProblemScans

• %Path.DataRoot%\Documents\ImportFailures

• Alerts are setup to trigger on items located here.

• \Attachments -Non-supported file types

• \Conversion -Problem converting to PDF

• \GlobalRouteFailure -Failed Global Routing

• \InsertFailure -Failed document record insert into db

• \Unknown -Catch all




File System Folder Structure, Business Units(continued)

Files Structures that have Business Unit specific context reside at:

• %Path.DataRoot%\BusinessUnits

Every DocuTrack install has the following BU file structures

• \Global -Globally applicable

• \Default -Default items used to create a new BU

• \1 -The first BU instance

• Further BU’s are assigned numbers sequentially \2,\3,\4…

Fax Cover Pages - (Global and BU Specific - cumulative)

• Path.DataRoot%\BusinessUnits\<context>\CoverPages

Forms – (Global and BU specific - cumulative)

• %Path.DataRoot%\BusinessUnits\<context>\Forms




File System Folder Structure, Business Units (continued)

Templates - (Global ONLY at this time – DT5.3.3)

• Path.DataRoot%\BusinessUnits\Global\Templates

• Fax Confirmation pages template

• EMAR templates

• Failed Fax notification (that users see in DT)




Integra DocuTrack Services

DocuTrack product Services (Integra DocuTrack ____ Service)

• Alert – Monitors for and generates Alerts – popups and emails – always running

• Document Archive – Runs the Archive jobs – always running

• Document Import – Imports documents – always running

• Document Processing – Document Import rules processing – always running

• Fax Service – Inbound and outbound faxing (with exceptions) – always running

• File Reader Service – Processing of file type interface data – if an interface is configured

• Hosting Service – WFC control service – always running

• Page Count Service (as of 5.3.2) – Page Audit reconciliation – only run once


An understanding of the services is important to understand DocuTrack logging



DocuTrack Logging (Server)

%Path.DataRoot%\Logs

• All DocuTrack logs reside in subfolders, by service name.

• Text files, but open nicely in tools like BareTail and BareGrep.

• Location is configurable, but not for the faint of heart.

Integra Logging Tools

• Start Menu, OR

• %PROGRAMFILES(x86)%\Integra\DocuTrack\bin\LogFilterTool.exe

• Demo




DocuTrack Logging (Client)

%ProgramFiles(x86)%\Integra\DocuTrack\Logs\DocuTrack.Client.<user>.log

• Text files, but open nicely in tools like BareTail and BareGrep.

• Location is configurable, but not for the faint of heart.

• Unique feature starting with DT 5.3.2 client, the username is embedded in the logfile name, making them unique per user. See Citirx.

Integra Logging Tool

• %PROGRAMFILES(x86)\Integra\DocuTrack\bin\LogFilterTool.exe

• RunAs Adminsitrator (recommended, depending on logged in users rights).

• Note in a shared environment, this will change logging level for all users.

• Demo




Other useful logs for review

Windows Event Logs

• In particular, look for .NET framework errors

MS SQL Logs

• SQL Server Management Studio (SSMS)

• SQL Server Agent, Error Logs (SSMS 2008R2)

MS SQL Job Activity Monitor (key SQL jobs)

• Status of SQL Agent Jobs

• Background Updates– should not be long running job, or failing

• Full Backup and Trans log backups should be succeeding, (or disabled if not used)



Faxing Solutions and what can typically failThe pieces that make DocuTrack work

FACSys FAX Server– Tools and Log locations

Facsys Desktop client

• Good for looking at inbound and outbound faxes over time

• Old Fax images are pruned after 14 days (configurable)

Logs

• D:\FACSys\Facsys services\LogFiles

• File name: fx<service><datecode>.log

• Service “m” = device manager service

• Service “r” = fax router service

Typical problems

• FACSys services have stopped or failed. Will happen when a NIC is added/removed/changed.

• SMTP connectivity between DT server and FACSys server is failing



Customer IT focusWhat should I be managing?

Data Backup and Retention

It is absolutely critical a backup and recovery strategy be built and regularly tested.

Key DocuTrack parts to backup. (required)

• %Path.DataRoot% - Document PDF Store and the SQL Backups

• Do not backup running SQL DB files. They are not restorable. (or use an Agent)

Any customized files/products outside of DocuTrack.

• FACSys – typically cover pages (D:\Facsys)

Best practices are what best serves the Pharmacy. (What would it cost to lose DT?)

• Rotate media daily

• Actively monitor Backup logs

• Retain a backup in an archive at least quarterly, monthly better.

• Test a full recovery at least annually, spot check at least monthly

• Dell recommends replacing tape media every 1000 hours of tape travel time.



Customer IT focus (cont.)What should I be managing?

Network Infrastructure and best practices.

If you have a separated SQL server, make sure connecting network is rock solid.

Understand and correctly implement your machine name resolution and IP address assignment technologies.

Safely store the DocuTrack server’s local NT Administrator account password. Make sure it is shared appropriately. Integra cannot provide it, we can only change it, and only if we have access to your server.

Make sure AD domain service account passwords are set to not expire, and do not have time of day restrictions.

Apply GPO’s to DocuTrack server only after testing on non-production box.

Fully document your firewall and VPN architectures and policies. Test before making changes.

Maintain a change control process for all critical pieces.

Source: DocuTrack Support experiences



3rd party software and services

Antivirus solution. Integra provisioned servers come with a 1 yr. subscription to AVG.

Image Printing (Zan Printer – www.zan1011.com).

• Pharmacy purchased direct from site. Maintain your licenses keys.

• Integra maintains documentation on how to configure for use with DocuTrack.

• Generally, we find problems related to Zan are either workstation setup or application XYZ printing through Zan.

Pop3 mail client (popclient.exe – www.codeode.com)

• Simply opens a connection to a pop3 mail account and downloads emails and drops them into a configured file location (import)

• free, no support provided by vendor.

• Integra maintains configuration files that work for DocuTrack

• You are free to investigate something more robust, there is nothing special about this application for DocuTrack.


http://www.zan1011.com/

http://www.codeode.com/



3rd party software and services (cont.)

Microsoft Windows Updates

• Integra tests every DocuTrack release with then current Security and Critical Windows updates installed on test environments (as of release date).

• Best practices, review all updates and understand each before applying.

• Maintain a change control (made easier with Windows Updates logging)

• Apply updates to a tests system first and verify basic OS/ boot functionality

• Consider the implications of being an early adopter. How long do you wait??

Dell Hardware warranty (www.dell.com)

• Integra sells all new servers with a 3 year, 4 hour response time warranty support, after which the Pharmacy is responsible for renewing.

• !! NEW this year – Dell will not take hardware related support calls without a warranty in force.

• Your Dell service tag and the web site is all you need to determine warranty status.



Getting the most out of your Support

DocuTrack Support options

eMail, Phone, or Web Form

• Choose your method based on the severity of the issue.

• Critical down situations should always be a phone call to Support.

• If you use email, always retain the ticket number in the email subject line on subsequent replies.

• Staffed hours phone calls are on average answered in less than 1 minute. I monitor this metric.

Standard vs. Extended Support

• Extended Support adds additional day/hours you can contact Support.

• Standard support customers are limited to staffed office hours, but can get help outside those times, on a per case basis, at then current Time and Material rates. To do so:

• Phone: Support number, use the OnCall menu option.

• eMail: Support address, use STAT! modifier on your auto-acknowledgement email.



Getting the most out of your Support (cont.)

DocuTrack/OS Upgrades

Update/Upgrade days/hours

• Monday – Friday, 5am – 12am PT, excluding holidays

• Outside those hours, at T&M rates, as staff scheduling permits.

DocuTrack Release awareness

• All updates are announced in the newsletter.

• Email to [email protected] to subscribe. See bottom of newsletter to unsubscribe.

• All releases have a Customer documentation bundle generated by DocuTrack Support, that is distributed to you upon opening a support ticket for an update or by request. If you read nothing else in that bundle, please read the FAQ. It contains cautions that may apply to you.

Workstation OS support

• If you are not already doing so, plan to replace WinXP in 2014.

• Avoid Vista, if at all possible.

• Test your applications with Win8.1 before upgrading. (DT 5.4.0 will support Win8.1)

Source: DocuTrack Support Procedures

mailto:[email protected]


Wrap-up / Questions / CommentsCan I demo anything for you on running system?

PowerPoint presentations will be available on the Integra Web site after User Conference

We are located in lovely Anacortes, Washington, the Gateway to the San Juan Islands. If ever in the Pacific Northwest, come visit!

DocuTrack Support:

866-720-6846 (Staffed Mon-Fri. 5am – 8pm PT, other days/hours via On Call)

Support email: [email protected]

Corporate web site: www.integragroup.com

My Email: [email protected]

Source: Integra Marketing materials


http://www.integragroup.com/



Thank You