View
2.594
Download
5
Category
Preview:
DESCRIPTION
Dynamically skinning a legacy portal using Python, WSGI (the Python Web Server Gateway Interface), and Deliverance. So you have a big legacy portal application which you want to change the look of, but are contractually not allowed to touch? Here is a case study on how we used the power and flexibility of Python and WSGI and the wonder lxml to dynamically re-skin a proprietary .NET portal without even touching it. We take a giant lump of messy invalid HTML markup and dynamically strip it back, add semantic markup and CSS and present the user with a nice svelte valid site. I will cover the history of the legacy portal, the problems encountered, our cunning plan to dynamically re-skin the site, a technical overview of the parts of the system (lxml, WSGI, etc), and what we learned along the way.
Citation preview
30th June 2009 Europython 2009, Birmingham, UK 1
Lipstick on a Pig
Dynamically re-skinning a legacy .NET portal with python
Matt Hamiltonmatth@netsight.co.uk
30th June 2009 Europython 2009, Birmingham, UK 2
Introduction
Dynamically re-skinning a .NET portal site Can't name the client Portal for teachers in the UK Aggregating content across legacy portal, Plone
and Moodle
30th June 2009 Europython 2009, Birmingham, UK 3
Who Am I
Technical Director of Netsight→ Web development firm in Bristol, UK
10 years experience with Zope/Plone More of an integrator than core developer
→ I get involved in all those sticky projects of merging Plone in with other systems in an enterprise
30th June 2009 Europython 2009, Birmingham, UK 4
Where Were We?
30th June 2009 Europython 2009, Birmingham, UK 5
Existing Portal (1.0)
Five years old by November 2009 User registrations: 46,681 Course enrolments: 33,664 Resource Bank views: 247,911
30th June 2009 Europython 2009, Birmingham, UK 6
Existing Portal (1.0)
30th June 2009 Europython 2009, Birmingham, UK 7
Problems with Current Portal Look-and-feel
→ Not very compelling Usability
→ Challenging in places A poor content management system
→ Can't really edit general content, so use a separate FTP server and Dreamweaver
Vendor lock-in→ Even small changes, very expensive
30th June 2009 Europython 2009, Birmingham, UK 8
The Future - Portal 2.0
Usability, Design and Content Review April 2008
Strategic Review August 2008 Feasibility Studies Jan 2009 Pilot Demonstrator (“Portal V1.5”) March 2009
30th June 2009 Europython 2009, Birmingham, UK 9
Design Review
30th June 2009 Europython 2009, Birmingham, UK 10
Architecture Review
Portal 1.0 - Monolithic, tightly coupled, poor separation of skin
30th June 2009 Europython 2009, Birmingham, UK 11
Architecture Review
Portal 2.0 - Extensible, loosely coupled, good separation of skin
30th June 2009 Europython 2009, Birmingham, UK 12
How Do We Get There?
Remember: We Can't Touch the Existing System!
30th June 2009 Europython 2009, Birmingham, UK 13
The Cunning Plan
Skin 1.0
Portal 1.0
ExistingPortal Module of
functionality e.g. portfolio
BrowserBrowser
30th June 2009 Europython 2009, Birmingham, UK 14
Total Skin Graft!
Portal 1.5New skin via
xpath and xslttransformation.
Web server needs to handle
SSL.
Together they give us nice
URLs.
Skin 1.0
web server
transformation proxySkin 2.0
BrowserBrowser
30th June 2009 Europython 2009, Birmingham, UK 15
Deliverance
Several Different Projects→ xdv→ Deliverance 0.3
To learn more on specifics of Deliverance go to http://deliverance.openplans.org
30th June 2009 Europython 2009, Birmingham, UK 16
WSGI
WSGI allows you to write small modules chained together in a 'pipeline'
Many small filters combined together as you need
Lots of existing components out there Very easy to write new ones
30th June 2009 Europython 2009, Birmingham, UK 17
WSGI Power - The Pipeline
[pipeline:portal]pipeline = theme.portal ploneinterceptor xslt linkrewrite htmlcleaner source.portal
[filter:theme.portal]use = egg:dv.xdvserver#xdvtheme_uri = file://%(here)s/theme/theme.htmlrules = %(here)s/rules/content.xml
[filter:ploneinterceptor]use = egg:ns.ploneinterceptor#ploneinterceptor
[filter:xslt]use = egg:dv.xdvserver#xsltxslt_file = %(here)s/rules/transform.xsl
[filter:linkrewrite]use = egg:ns.linkrewrite#linkrewrite
[filter:htmlcleaner]use = egg:ns.htmlcleaner#htmlcleaner
[app:source.portal]use = egg:Paste#proxyaddress = http://www.theexistingsite.org.uk/
30th June 2009 Europython 2009, Birmingham, UK 18
Link Rewriting
Old URL:→ http://www.theclient.org.uk/WebPortal.aspx?page=1&module=DB920A53-01EA-4886-8878-F2CDF5FA8CFD&mode=101&IsNonNewsDB920A53_01EA_4886_8878_F2CDF5FA8CFD=True&newsIdDB920A53_01EA_4886_8878_F2CDF5FA8CFD=11208#10
→ 205 characters! New URL:
→ http://www.theclient.org.uk/news/11208#10→ 41 characters!
30th June 2009 Europython 2009, Birmingham, UK 19
HTML Cleanup
LXML rules!→ from lxml.html.clean import Cleanercleaner = Cleaner(...)# Pretty print the HTMLdom = document_fromstring(body)body = etree.tostring(dom, pretty_print=True)# Clean the HTMLbody = cleaner.clean_html(body)
30th June 2009 Europython 2009, Birmingham, UK 20
Result
Old→ 70kb of HTML→ 120 Validation errors, 61 warnings
New→ 40Kb of HTML→ 27 Errors, 1 warning (mainly xhtml/html conflicts)→ No significant performance impact
30th June 2009 Europython 2009, Birmingham, UK 21
Putting all togetherComposite:main
pipeline:portal pipeline:plone pipeline:moodle
theme.content
navmerger
plonecontent
source.plone
theme.content
xslt
linkrewrite
htmlcleaner
theme.content
navmerger
moodlecontent
source.moodle
source.portal
30th June 2009 Europython 2009, Birmingham, UK 22
End ResultNew Style
Portal content
30th June 2009 Europython 2009, Birmingham, UK 23
Complications
Navigation→ One page, two content sources, how is the
navigation built? Search
→ Search needs to go across multiple systems→ Will soon be looking at Solr, Xapian, Google Mini
.NET viewstate postback→ Massive hidden state variable, form wraps entire
site!
30th June 2009 Europython 2009, Birmingham, UK 24
Questions?
matth@netsight.co.uk
30th June 2009 Europython 2009, Birmingham, UK 25
We are looking for Developers!
Come chat to me or
drop an email to
careers@netsight.co.uk
Recommended