Oops, I broke my API

© 2016, iText Group NV© 2016, iText Group NV

Oops, I broke my APIBruno Lowagie & Raf Hens: iText @ JavaOne 2016@bruno1970 – @rafhens – @iText

© 2016, iText Group NV

2 Oops, I broke my API - JavaOne 2016: @bruno1970, @rafhens, @iText

Agenda: five important questions

Who? Who are we and how did we break our APIWhat? What does it mean when people say they broke their APIWhy? What are acceptable reasons to break an APIHow? Or rather: how not to break your APIWhen? When to break the API, once the decision is made to do it


Who?

Who are we? What inspired us to present this talk? What do we want to achieve?


Oops, I broke my API - JavaOne 2016: @bruno1970, @rafhens, @iText4

Bruno LowagieOriginal developer of iTextex-CEO, current CTO at iText Group

Raf HensDirector of Engineering at iText GroupProject lead of iText 7

Who are we?



What is iText?iText is an open source library for creating, processing, and manipulating PDF files in Java and .NET.



We broke our API (more than once)2000: iText 0.30; first release (LGPL, later MPL/LGPL)• 2003: iText 1.00• 2007: iText 2.0.0• 2009: iText 2.1.7; last version available under MPL/LGPL

2009: iText 5.0.0; API deliberately broken (switch to AGPL)• 2012: iText 5.2.0; broken release (undeliberate)• 2013: iText 5.3.x; deliberate API change for digital signature functionality• 2016: iText 5.5.9; last “backward compatible” version

2016: iText 7; rewritten from scratch, not backward compatible



We’re not aloneIn this talk, we want to share our experience so that other developers can avoid making the mistakes we made. We’ll also show other examples of APIs that were broken and explain what we learned from those examples


What?

What does “breaking an API” actually mean? Binary compatibility Source code compatibility Behavioral compatibility



Backwards compatibility

What is it?

• Java: original specification still 100% valid• PDF: PDF 1.0 should still render 100% correctly

Examples of backwards compatibility



Three types of compatibility

• a certain type of compiled file can run on both systems (whether it be hardware or software) without any changes

Binary compatibility

• the input file must be recompiled for every system, but does not need to be changed before compiling.

Source code compatibility

• the behavior when processing the input file is the same

Behavioral compatibility


Why?

What are acceptable reasons to break an API? Why did we break our API?



Involuntary API breaks

• Customized version of Bouncy Castle in Android• Renamed version “Spongy Castle” to work around this issue

Bouncy Castle

• A third party created a fork: iText 4• The official iText groupID was used (which is not allowed)• This broke many Maven builds

iText



Avoid breaking the API when possible Use internal re-implementations Use two methods for the same function Deprecate old methods

Redundancy!



Redundancy can get painfulRationale for deprecating XFA

Rationale for Python 3

• HTMLWorker / XML Worker / …• Rendering APIs iText 5 / iText 7

Rationale for iText 7



In response to advancing technology

• Example: PDF

Standards and specifications change

• Hashing and encryption algorithms• Impact on digital signatures

Processing power increases

• ASCII isn’t sufficient anymore, we need Unicode

Internationalization



Changing the API as a strategy

• Old games don’t work on new consoles

PlayStation, Xbox

• Reduce server load• Reduce abuse

Twitter API change 2013

• Replace competing add-on with own add-on

SharePoint 2013



Why we changed the iText design

• The core design hadn’t changed since 2000• Organic growth; many different contributors• The world has changed since 2000

iText 7 versus iText 5


How?

What should you do to limit the damage? What should you avoid?



Use semantic versioning

Use an x.y.z notation

• The major version number (x):• for breaking changes

• The minor version number (y):• for new, yet backwards

compatible, features• The patch version number (z):

• for updates which fix bugs

How not to do it:

• BouncyCastle• PHP



A good design to start with is key

Spring 2 and Spring 3

• The original design was future-proof

Python 2 and Python 3

• It took 4 years to build Python 3 (2004-2008)

• Adoption of Python 3 was extremely slow



Avoid being a moving target

• Switch from Java 5 to Java 7• without changing the version (1.0.0-SNAPSHOT)

Commons-imaging

• No leadership: breaking changes in minor versions

PHP

• Developed internally until API was more or less stable

iText 7



Use tools to detect breaking changes Clirr: http://clirr.sourceforge.net/

Maven plugin (mvn clirr:check) We run it in a separate profile

SonarQube plugin Tip: create a separate SQ dashboard

JDiff: http://javadiff.sourceforge.net/ Javadoc doclet Generates HTML report of API differences

http://clirr.sourceforge.net/

http://javadiff.sourceforge.net/



Ease the pain

• iText: tutorials on Leanpub

Provide good documentation

• Python: tools• Microsoft Word: document format conversion

Provide conversion tools



Avoid back-porting

• Disadvantages of back-porting

Python 3

• No new functionality in iText 5 (e.g. PDF 2.0)

iText 7



Have a clear EOL strategy

Python 2 procrastination

• Constantly changing EOL date

iText

• iText 5: December 31, 2018

• iText 7: December 31, 2025


When?

Which conditions should be met before breaking the API? When is it acceptable to make a release that breaks the API?



It’s marketing’s fault

Product is announced before it’s ready• Example: the Universal

Windows PlatformResult: confusion?!



It’s the sales’ fault

Want to sell an upgrade NOW

Pushing to release new versions

Don’t want to lose business

Putting on hold communication about new versions to keep current business old

version

Result: confusion



In any case…

• Deprecate before you remove a class or a method• Warn when code is subject to change

Don’t let it happen unnoticed

• A license change also breaks the API

Beware of non-technical changes


Use case: iText 7

New insights Lessons Learned



Making development future-proof

New insights that led to a complete rewrite of iText 7

Make it modular, make it extensible

Remove functional overlap to avoid

maintenance hell

Meet customer requests, e.g. special

writing systems



Big bang release or early release?Big bang: what if we got it wrong?

Early release: heavy burden on support

• Release early for selected users• Release as open source

• As soon as the API is stable• Before the work is completely done

Something in-between:


Questions?



Hello World: iText 5public void createPdf(String dest) throws DocumentException, IOException { Document document = new Document(); PdfWriter.getInstance( document, new FileOutputStream(dest)); document.open(); document.add(new Paragraph("Hello World!")); document.close();}



Hello World: iText 7 (example 1)public void createPdf(String dest) throws IOException { PdfDocument pdf = new PdfDocument(new PdfWriter(dest)); Document document = new Document(pdf); document.add(new Paragraph("Hello World!")); document.close();}



Hello World: iText 7 (example 2)public void createPdf(String dest) throws IOException { PdfDocument pdf = new PdfDocument(new PdfWriter(dest)); try (Document document = new Document(pdf)) { document.add(new Paragraph("Hello World!")); }}



Hello World: iText 5 vs iText 7



Fonts: iText 5Font font = new Font(FontFamily.TIMES_ROMAN);Font font14pt = new Font(FontFamily.TIMES_ROMAN, 14);Font font10pt = new Font(FontFamily.TIMES_ROMAN, 10);BaseFont bf_russian = BaseFont.createFont( "resources/fonts/FreeSans.ttf", "CP1251", BaseFont.EMBEDDED);Font russian = new Font(bf_russian, 12);BaseFont bf_cjk = BaseFont.createFont( "resources/fonts/NotoSansCJKsc-Regular.otf", BaseFont.IDENTITY_H, BaseFont.EMBEDDED);Font cjk = new Font(bf_cjk, 12);



Fonts: iText 7PdfFont font = PdfFontFactory.createFont(FontConstants.TIMES_ROMAN);PdfFont russian = PdfFontFactory.createFont( "src/main/resources/fonts/FreeSans.ttf", "CP1251", true);PdfFont cjk = PdfFontFactory.createFont( "src/main/resources/fonts/NotoSansCJKsc-Regular.otf", PdfEncodings.IDENTITY_H, true);



Fonts: iText 5Paragraph p = new Paragraph("Hello World! ", font);Chunk chunk = new Chunk("Hallo Wereld! ", font14pt);p.add(chunk);chunk = new Chunk("Bonjour le monde! ", font10pt);chunk.setTextRise(4);p.add(chunk);chunk = new Chunk("\u0417\u0434 ... \u0440! ", russian);p.add(chunk);p.add(new Chunk("\u4f60\u597d\u4e16\u754c! ", cjk));p.add(new Chunk("\uc5ec\ubcf4 ... \uacc4!", cjk));document.add(p);



Fonts: iText 7document.setFont(font);Paragraph p = new Paragraph("Hello World! ") .add(new Text("Hallo Wereld! ") .setFontSize(14)) .add( new Text("Bonjour le monde! ") .setFontSize(10) .setTextRise(4)) .add( new Text ("\u0417\u0434 ... \u0440! “).setFont(russian)) .add( new Text("\u4f60\u597d\u4e16\u754c! ").setFont(cjk)) .add( new Text("\uc5ec\ubcf4 ... \uacc4!").setFont(cjk));document.add(p);



Fonts: iText 5 vs iText 7



Switching styles: iText 5public Chunk createBgChunk(String s, Font font) { Chunk chunk = new Chunk(s, font); chunk.setBackground(BaseColor.LIGHT_GRAY); return chunk;}Font code = new Font( FontFamily.COURIER, 12, Font.NORMAL, BaseColor.RED);Paragraph p = new Paragraph("In this example, named ");p.add(createBgChunk("HelloWorldStyles", code));p.add(", we experiment with some text in ");p.add(createBgChunk("code style", code));p.add(".");



Switching styles: iText 7Style style = new Style() .setFont(code) .setFontSize(12) .setFontColor(Color.RED) .setBackgroundColor(Color.LIGHT_GRAY);document.add( new Paragraph() .add("In this example, named ") .add(new Text("HelloWorldStyles").addStyle(style)) .add(", we experiment with some text in ") .add(new Text("code style").addStyle(style)) .add("."));



Switching styles: iText 5 vs iText 7



Tables: iText 5PdfPTable table = new PdfPTable(3);

PdfPCell cell = new PdfPCell(new Phrase("Cell with colspan 3"));cell.setColspan(3);cell.setHorizontalAlignment(Element.ALIGN_CENTER);table.addCell(cell);

cell = new PdfPCell(new Phrase("Cell with rowspan 2"));cell.setRowspan(2);cell.setVerticalAlignment(Element.ALIGN_MIDDLE);table.addCell(cell);



Tables: iText 5// text modetable.addCell("Cell 1.1");

// composite modecell = new PdfPCell();cell.addElement(new Phrase("Cell 1.2"));table.addCell(cell);



Tables: iText 5cell = new PdfPCell(new Phrase("Cell 2.1"));cell.setPadding(5);cell.setUseAscender(true);cell.setUseDescender(true);cell.setHorizontalAlignment(Element.ALIGN_CENTER);table.addCell(cell);cell = new PdfPCell();cell.setPadding(5);cell.setUseAscender(true);cell.setUseDescender(true);Paragraph p = new Paragraph("Cell 2.2");p.setAlignment(Element.ALIGN_CENTER);cell.addElement(p);table.addCell(cell);



Tables: iText 7Table table = new Table(3);Cell cell = new Cell(1, 3) .setTextAlignment(TextAlignment.CENTER) .add("Cell with colspan 3");table.addCell(cell);cell = new Cell(2, 1) .add("Cell with rowspan 2") .setVerticalAlignment(VerticalAlignment.MIDDLE);table.addCell(cell);



Tables: iText 7table.addCell("Cell 1.1");table.addCell(new Cell().add("Cell 1.2"));table.addCell(new Cell() .add("Cell 2.1") .setBackgroundColor(Color.LIGHT_GRAY) .setMargin(5));table.addCell(new Cell() .add("Cell 1.2") .setBackgroundColor(Color.LIGHT_GRAY) .setPadding(5));



Tables: iText 5 vs iText 7



TXT 2 PDF: iText 5Document document = new Document();PdfWriter.getInstance(document, new FileOutputStream(dest));document.open();BufferedReader br = new BufferedReader(new FileReader(TEXT));String line;Paragraph p;Font normal = new Font(FontFamily.TIMES_ROMAN, 12);Font bold = new Font(FontFamily.TIMES_ROMAN, 12, Font.BOLD);boolean title = true;while ((line = br.readLine()) != null) { p = new Paragraph(line, title ? bold : normal); p.setAlignment(Element.ALIGN_JUSTIFIED); title = line.isEmpty(); document.add(p);}document.close();



TXT 2 PDF: iText 7PdfDocument pdf = new PdfDocument(new PdfWriter(dest));Document document = new Document(pdf) .setTextAlignment(TextAlignment.JUSTIFIED);BufferedReader br = new BufferedReader(new FileReader(TEXT));String line;PdfFont normal = PdfFontFactory.createFont(FontConstants.TIMES_ROMAN);PdfFont bold = PdfFontFactory.createFont(FontConstants.TIMES_BOLD);boolean title = true;while ((line = br.readLine()) != null) { document.add(new Paragraph(line).setFont(title ? bold : normal)); title = line.isEmpty();}document.close();



TXT 2 PDF: iText 5 vs iText 7



Columns: iText 5Document document = new Document();PdfWriter writer = PdfWriter.getInstance(document, new FileOutputStream(dest));document.open();ColumnText ct = new ColumnText(writer.getDirectContent());



Columns: iText 5BufferedReader br = new BufferedReader(new FileReader(TEXT));String line;Paragraph p;Font normal = new Font(FontFamily.TIMES_ROMAN, 12);Font bold = new Font(FontFamily.TIMES_ROMAN, 12, Font.BOLD);boolean title = true;while ((line = br.readLine()) != null) { p = new Paragraph(line, title ? bold : normal); p.setAlignment(Element.ALIGN_JUSTIFIED); title = line.isEmpty(); ct.addElement(p);}



Columns: iText 5Rectangle[] columns = { new Rectangle(36, 36, 290, 806), new Rectangle(305, 36, 559, 806) }int c = 0;int status = ColumnText.START_COLUMN;while (ColumnText.hasMoreText(status)) { ct.setSimpleColumn(columns[c]); status = ct.go(); if (++c == 2) { document.newPage(); c = 0; }}document.close();



Columns: iText 7PdfDocument pdf = new PdfDocument(new PdfWriter(dest));Document document = new Document(pdf) .setTextAlignment(TextAlignment.JUSTIFIED);Rectangle[] columns = { new Rectangle(36, 36, 254, 770), new Rectangle(305, 36, 254, 770) };document.setRenderer(new ColumnDocumentRenderer(document, columns));BufferedReader br = new BufferedReader(new FileReader(TEXT));String line;PdfFont normal = PdfFontFactory.createFont(FontConstants.TIMES_ROMAN);PdfFont bold = PdfFontFactory.createFont(FontConstants.TIMES_BOLD);boolean title = true;while ((line = br.readLine()) != null) { document.add(new Paragraph(line).setFont(title ? bold : normal)); title = line.isEmpty();}document.close();



Columns: iText 5 vs iText 7



Page events: iText 5Document document = new Document();PdfWriter writer = PdfWriter.getInstance(document, new FileOutputStream(dest));MyPageEvents events = new MyPageEvents();writer.setPageEvent(events);document.open();



Page events: iText 5BufferedReader br = new BufferedReader(new FileReader(TEXT));String line;Paragraph p;Font normal = new Font(FontFamily.TIMES_ROMAN, 12);Font bold = new Font(FontFamily.TIMES_ROMAN, 12, Font.BOLD);boolean title = true;while ((line = br.readLine()) != null) { p = new Paragraph(line, title ? bold : normal); p.setAlignment(Element.ALIGN_JUSTIFIED); events.setTitle(title); document.add(p); title = line.isEmpty();}document.close();



Page events: iText 5class MyPageEvents extends PdfPageEventHelper { protected float startpos = -1; protected boolean title = true;

public void setTitle(boolean title) { this.title = title; } @Override public void onEndPage(PdfWriter writer, Document document) { ... } @Override public void onParagraph(PdfWriter writer, Document document, float paragraphPosition) { ... } @Override public void onParagraphEnd(PdfWriter writer, Document document, float paragraphPosition) { ... }}



Page events: iText 5@Overridepublic void onEndPage(PdfWriter writer, Document document) { Rectangle pagesize = document.getPageSize(); ColumnText.showTextAligned( writer.getDirectContent(), Element.ALIGN_CENTER, new Phrase(String.valueOf(writer.getPageNumber())), (pagesize.getLeft() + pagesize.getRight()) / 2, pagesize.getBottom() + 15, 0); if (startpos != -1) onParagraphEnd(writer, document, pagesize.getBottom(document.bottomMargin())); startpos = pagesize.getTop(document.topMargin());}



Page events: iText 5@Overridepublic void onParagraph(PdfWriter writer, Document document, float paragraphPosition) { startpos = paragraphPosition;}@Overridepublic void onParagraphEnd(PdfWriter writer, Document document, float paragraphPosition) { if (!title) return; PdfContentByte canvas = writer.getDirectContentUnder(); Rectangle pagesize = document.getPageSize(); canvas.saveState(); canvas.setColorStroke(BaseColor.BLUE); canvas.rectangle( pagesize.getLeft(document.leftMargin()), paragraphPosition - 3, pagesize.getWidth() - document.leftMargin() - document.rightMargin(), startpos - paragraphPosition); canvas.stroke(); canvas.restoreState();}



Event handlers: iText 7PdfDocument pdf = new PdfDocument(new PdfWriter(dest));pdf.addEventHandler(PdfDocumentEvent.END_PAGE, new Footer());Document document = new Document(pdf).setTextAlignment(TextAlignment.JUSTIFIED);BufferedReader br = new BufferedReader(new FileReader(TEXT));String line;PdfFont normal = PdfFontFactory.createFont(FontConstants.TIMES_ROMAN);PdfFont bold = PdfFontFactory.createFont(FontConstants.TIMES_BOLD);boolean title = true;Border border = new SolidBorder(Color.BLUE, 1);while ((line = br.readLine()) != null) { document.add(new Paragraph(line) .setFont(title ? bold : normal) .setBorder(title ? border : Border.NO_BORDER)); title = line.isEmpty();}document.close();



Event handlers: iText 7protected class Footer implements IEventHandler { @Override public void handleEvent(Event event) { PdfDocumentEvent docEvent = (PdfDocumentEvent) event; PdfDocument pdf = docEvent.getDocument(); PdfPage page = docEvent.getPage(); Rectangle pageSize = page.getPageSize(); PdfCanvas pdfCanvas = new PdfCanvas( page.getLastContentStream(), page.getResources(), pdf); Canvas canvas = new Canvas(pdfCanvas, pdf, pageSize); float x = (pageSize.getLeft() + pageSize.getRight()) / 2; float y = pageSize.getBottom() + 15; canvas.showTextAligned( String.valueOf(pdf.getPageNumber(page)), x, y, TextAlignment.CENTER); }}



Events: iText 5 vs iText 7



Renderers: iText 7public class TitleParagraph extends Paragraph { public TitleParagraph(String line) { super(line); try { setFont(PdfFontFactory.createFont(FontConstants.TIMES_BOLD)); } catch (IOException ioe) { } } @Override protected IRenderer makeNewRenderer() { return new ParagraphRenderer(this) { ... }; } }



Renderers: iText 7@Overridepublic void drawBorder(DrawContext drawContext) { Rectangle occupiedAreaBBox = getOccupiedAreaBBox(); float[] margins = getMargins(); Rectangle rectangle = applyMargins(occupiedAreaBBox, margins, false); PdfCanvas canvas = drawContext.getCanvas(); canvas.roundRectangle(rectangle.getX() - 1, rectangle.getY() - 1, rectangle.getWidth() + 2, rectangle.getHeight() + 2, 5).stroke();}



Renderers: iText 5 vs iText 7



Form creation: iText 5PdfReader reader = new PdfReader(src);PdfStamper stamper = new PdfStamper(reader, new FileOutputStream(dest));TextField tf = new TextField(stamper.getWriter(), new Rectangle(110, 780, 180, 806), "text");tf.setBorderColor(BaseColor.BLUE);tf.setBorderWidth(2);tf.setTextColor(BaseColor.RED);tf.setFontSize(12);tf.setText("Text field");PdfFormField field = tf.getTextField();stamper.addAnnotation(field, 1);stamper.close();reader.close();



Form creation: iText 7PdfReader reader = new PdfReader(src);PdfDocument pdf = new PdfDocument(reader, new PdfWriter(dest));PdfAcroForm form = PdfAcroForm.getAcroForm(pdf, true);PdfFormField tf = PdfTextFormField.createText(pdf, new Rectangle(110, 780, 70, 26), "text", "Text Field") .setBorderColor(Color.BLUE) .setBorderWidth(2) .setColor(Color.RED) .setFontSize(12);form.addField(tf);pdf.close();



Form creation: iText 5 vs iText 7



Form filling: iText 5PdfReader reader = new PdfReader(src);PdfStamper stamper = new PdfStamper(reader, new FileOutputStream(dest));AcroFields fields = stamper.getAcroFields();fields.setFieldProperty("text", "textcolor", BaseColor.BLUE, null);fields.setFieldProperty("text", "bordercolor", BaseColor.RED, null);fields.setFieldProperty("text", "fontsize", 14, null);fields.setField("text", "Field Text");stamper.close();reader.close();



Form filling: iText 7PdfReader reader = new PdfReader(src);PdfDocument pdf = new PdfDocument(reader, new PdfWriter(dest));PdfAcroForm form = PdfAcroForm.getAcroForm(pdf, true);PdfFormField tf = form.getFormFields().get("text");tf.setBorderColor(Color.RED) .setColor(Color.BLUE) .setFontSize(14) .setValue("Field Text");pdf.close();



Form filling: iText 5 vs iText 7

Software

Oops, I broke my API