From Renamer Plugin to Polyglot IDE

From Renamer Plugin to Polyglot IDE

Dmitry JemerovCTO, JetBrains

Monday, September 9, 13

IntelliJ IDEA: 2000-2013

• Started as a plugin for JBuilder

• Currently a product line of 8 IDEs, a compiler, a DSL workbench and a server-side code browser

• $xxK initial investment, $yyyM total revenue

• HEAD is a usable IDE every single day


Agenda

• IntelliJ IDEA over the years

• IntelliJ IDEA development practices


IntelliJ IDEA:Over the Years


2000

• February 1st - company birthday

• Founded by Sergey Dmitriev, Eugene Belyaev and Valentin Kipiatkov from TogetherSoft

• IntelliJ Renamer, IntelliJ CodeSearch



Vista 1.0, Jan 2001

• PSI, VFS, commands

• Saved 2 months by not having plugin API

• Mentioned by Martin Fowler onhttp://refactoring.com/


http://refactoring.com

http://refactoring.com

Stella2.0, June 2001

• First external developers

• JSP, CVS, Ant, formatter, live templates


Pandora2.5, Dec 2001

• 13 new refactorings

• JUnit integration, one unit test in code

• Released 1 month after Eclipse 1.0


IDEA 2.6June 2002

• Company renamed to JetBrains

• JOLT Award in April 2002


Ariadna3.0, Nov 2002

• OpenAPI,

• 2 plugins (open-source): Starteam, Tomcat

• XML

• Real tests

• Oldest version available for download


Fabrique


Fabrique

• Framework and set of components for developing Web applications


Fabrique


• Visual IDE based on top of IntelliJ IDEA


Fabrique



• Drove a lot of platform API changes


Fabrique




• Project view, structure view, extensions


Fabrique




• Project view, structure view, extensions

• Canceled in 2005 before reaching 1.0



ReSharper

• Started in mid-2003

• Implemented in C#, di!erent architecture

• Initially used some parser/PSI technology from IntelliJ IDEA


Aurora4.0, Feb 2004

• Multiple-module projects

• On-the-fly inspections

• UI Designer


Pallada4.5, July 2004

• J2EE

• First two community-developed plugins

• Inspection Gadgets

• Intention PowerPack


Irida5.0, Aug 2005

• Custom language API

• JavaScript, Python

• Perforce + Subversion, open-source plugins

• 1M LOC

• 10 developers, no QA engineers



Demetra6.0, Oct 2006

• Core and Enterprise subteams

• TeamCity 1.0

• First plugin contest


Selena7.0, Oct 2007

• New caching VFS implementation

• Facets

• Ruby, Groovy


Diana8.0, Nov 2008

• Java-independent IntelliJ Platform extracted

• RubyMine 1.0 in April 2009

• Language-independent indices

• Language-independent debugger


Community EditionOct 2009

• Moved to git

• ~ 60% of codebase open-sourced

• Expected 30% drop in sales, got small gain

• Couple dozen external contributors


Maia9.0, Dec 2009

• Background indexing

• Artifacts

• PHP


Idea X, Xena10.0, Dec 2010; 10.5; Feb 2011

• Autopopup completion

• Android in Community Edition

• PhpStorm (May 2010), PyCharm (Oct 2010)


Nika11.0, Dec 2011

• UI redesign

• "core" package for Kotlin compiler

• AppCode (Oct 2011)


Leda12.0, Dec 2012

• Darcula

• External make

• UpSource, headless indexing framework


Android StudioMay 2013



• Built by Google with support by JetBrains




• Apache 2.0 licensed, no contracts and no money involved




• Apache 2.0 licensed, no contracts and no money involved

• 500K downloads in first 3 weeks


CardeaVersion 13, in development



• ~ 25 developers



• ~ 25 developers

• ~ 5.6M LOC, ~3M LOC open-source



• ~ 25 developers


• ~ 100K daily active users, 72% Ultimate



• ~ 25 developers


• ~ 100K daily active users, 72% Ultimate

• 50% Windows, 30% Mac, 20% Linux


IntelliJ IDEA:Development Practices


Break the Rules

• No detailed planning

• No unit tests

• No QA

• No code comments or internal docs

• No API compatibility

• Many wheels reinvented


Release Planning


Release Planning

• Management sets only high-level goals and target date


Release Planning


• Each developer responsible for detailed planning of their subsystem(s)


Release Planning



• No iteration planning


Release Planning



• No iteration planning

• No feature specifications


Standup Meetings

• Daily, broken into sub-teams

• Over video conference between St.Petersburg, Munich and Prague

• Allow management, QA and writers to stay on top of dev activity


Development Flow

• Everything done in master

• Almost no long-lived feature branches

• Branches used only for releases

• Refactorings in incremental steps

• New features side by side with existing code, turned on by system property


Automated Testing

• Mostly data-driven acceptance tests

• Code before, action to perform, code after

• Few pure unit tests, little usage of mocks

• Test framework agnostic (JUnit, TestNG, Cucumber)


Continuous Integration

• Used TeamCity since day 0, CruiseControl before that

• Remote run not mandatory but recommended


Testing: Upsides

• Easy to write

• Often just copy code example from bug report

• Very little fragility when impl changes

• Tests written 8 years ago still valuable


Testing: Downsides

• Whole test suite (37K tests) takes 7 hours to run

• Multiple commits per test run

• Failures di"cult to debug

• Especially async code (indexing, UI)

• Tests stay red for weeks


Testers

• Had no testers until 2006

• Focus on manual testing and usability

• No "quality assurance" as such, relying more on CI tests and user feedback


update.bat

• Every developer starts their day with building IntelliJ IDEA from latest sources

• "Do not update" emails if something badly broken

• Core Java stu! always works


Early Access Preview

• Public free-to-use builds released every 1-2 weeks

• Broad community testing for features we don't use internally

• A few thousand active EAP users

• Licenses for most helpful participants


Public Issue Tracker

• Initially ITN, then JIRA, then YouTrack

• ~ 50 new issues per day

• Low noise but many duplicates

• Triaging incoming issues – almost full-time job

• Imbalance between developers


Exception Analyzer

• Separate from issue tracker

• Reports grouped into problems

• Semiautomatic merging of duplicates

• Exception duty rotated between developers, takes a few hours per day


Code Review

• Previously used manual review (mostly face-to-face) for merging into release branches

• Now reviewing all platform changes

• Using Crucible and hating it

• Need tools to see context of change


Support


Support

• Until recently one support engineer (Serge Baranov) was covering all IntelliJ Platform-based IDEs


Support


• Best Technical Support (small to medium-sized business, SD Magazine Reader's Choice 2005)


Support


• Best Technical Support (small to medium-sized business, SD Magazine Reader's Choice 2005)

• Developers actively involved


Internal Docs


Internal Docs

• XP's belief: comments are a code smell


Internal Docs


• Most 3rd party plugins are open-source, good examples


Internal Docs



• Improving docs does not increase average plugin quality


Internal Docs



• Improving docs does not increase average plugin quality

• Investing into docs to promote IntelliJ Platform to companies


Plugin API


Plugin API

• No separate facade, plugins access internal IDE classes directly


Plugin API

• No separate facade, plugins access internal IDE classes directly

• No way to repurpose IDE the way we did without breaking API compatibility


Community Plugins

• Large part of feature set

• Always by agreement with author

• Usually reworked at JetBrains

• Usually kept open-source

• Rewards through plugin contest

• Sometimes hiring authors


Build System

• Not using Maven or Gradle

• Still storing dependency .jar files in VCS

• JPS: tool that builds IntelliJ IDEA project from command line

• Same as external make in IDE

• Gant scripts to generate distribution


Extensibility

• Not using OSGi, Guice or Spring

• PicoContainer for dependency injection

• Home-grown extension point system

• Components and extensions to load specified in .xml files


Non-Java Languages

• Some Groovy for tests

• Live Edit's Chrome extension written in Kotlin and translated to JavaScript

• Scala plugin written in Scala

• Clojure plugin uses some Clojure


Summary


Summary

• Codebase repurposed far beyond original goals through relentless refactoring


Summary


• Full-time dogfooding is essential for maintaining quality and usability


Summary


• Full-time dogfooding is essential for maintaining quality and usability

• Lightweight process is enough for product development with no external stakeholders


Q&A

[email protected]@intelliyole

Develop with Pleasure!


mailto:[email protected]

mailto:[email protected]

Technology

From Renamer Plugin to Polyglot IDE