View
1.368
Download
10
Category
Tags:
Preview:
DESCRIPTION
Writing REST APIs with ORMs and web frameworks is a chore. I'm lazy, and I don't want to write boring code. In this talk, I'll go over what REST APIs are, why they're useful, and why we should never have to write one from scratch again. By the end of this talk, we'll have achieved developer Nirvana: a RESTful API service and Admin interface for existing databases *without writing any code*.
Citation preview
JeffKnupp@jeffknuppjeff@jeffknupp.comWhartonWebConference2014
Authorof“WritingIdiomaticPython”Full-timePythondeveloper@AppNexusBloggeratjeffknupp.comCreatorofthe“sandman”Pythonlibrary
We'regoingtousePythonto generateaRESTAPI.
Andwe'regoingtodoitwithoutwritingasinglelineofcode.
We'llgooverwhataRESTAPIis,howitworks,andwhyit'susefulWe'llreviewtheHTTPprotocolandhowthewebworksWe'llseealotofPythoncode
Sevenletters.Twoacronyms.Buywhatdoesitmean?
Programmaticwayofinteractingwithathird-partysystem.
WaytointeractwithAPIsoverHTTP(thecommunicationprotocoltheInternetisbuilton).
"REST"wascoinedbyRoyFieldinginhis2000doctoraldissertation.Includessetofdesignprinciplesandbestpracticesfordesigningsystemsmeanttobe"RESTful".
InRESTfulsystems,applicationstateismanipulatedbytheclientinteractingwithhyperlinks.Arootlink(e.g.
)describeswhatactionscanbetakenbylistingresourcesandstateashyperlinks.
http://example.com/api/
HTTPisjustamessagingprotocol.HappenstobetheonetheInternetisbasedon.
RESTfulsystemsusethisprotocoltotheiradvantagee.g.cachingresources
GETPOSTPUTPATCHDELETE
TounderstandhowRESTAPIswork,wehavetounderstandhowthewebworks.
EverythingyouseeonthewebistransferredtoyourcomputerusingHTTP.
Whathappenswhenwetypehttp://www.jeffknupp.comintoourbrowser?
Let'stracethelifecycleofabrowser'srequest.
AprotocolcalledtheDomainNameService(DNS)isusedtofindthe"real"(IP)addressofjeffknupp.com.
GET
ThebrowsersendsaGETrequestto192.168.1.1forthepageataddress/(thehomeor"root"page).
The (aprogramusedtoserviceHTTPrequeststoawebsite)receivestherequest,findstheassociatedHTMLfile,andsendsitasanHTTPResponse.
Ifthereareanyimages,videos,orscriptsthattheHTMLmakesreferenceto,separateHTTPGETrequestsaremade
forthoseaswell.
Programs,likecurl,canalsoissueHTTPrequests
CURL
curltalkstothewebserver,usingapublicAPI(viaHTTP)
ARESTAPIexposesyourinternalsystemtotheoutsideworld
It'salsoafantasticwaytomakeasystemavailabletoother,internalsystemswithinanorganization.
ExamplesofpopularRESTAPIs:TwitterGitHubGoogle(foralmostallservices)
Ifyou'reaSaaSprovider,youareexpectedtohaveaRESTAPIforpeopletowriteprogramstointeractwithyour
service.
FourcoreconceptsarefundamentaltoallRESTservices(courtesyWikipedia)
WhenusingHTTP,thisisdoneusingaURI.Importantly,aresourceand arecompletelyorthogonal.Theserverdoesn'treturndatabaseresultsbutratherthe
JSONorXMLorHTMLrepresentationoftheresource.
Whentheservertransmitstherepresentationoftheresourcetotheclient,itincludesenoughinformationforthe
clienttoknowhowtomodifyordeletetheresource.
Eachrepresentationreturnedbytheserverincludesinformationonhowtoprocessthemessage(e.g.usingMIME
types
Clientsare .Theyknownothingabouthowtheserviceislaidouttobeginwith.Theydiscoverwhatactionstheycan
takefromtherootlink.Followingalinkgivesfurtherlinks,definingexactlywhatmaybedonefromthatresource.
Clientsaren'tassumedtoknow exceptwhatthemessagecontainsandwhattheserveralreadytoldthem.
ARESTAPIallows tosend tomanipulate .
...SoweneedtowriteaservercapableofacceptingHTTPrequests,actingonthem,andreturningHTTPresponses.
Yep.ARESTfulAPIServiceisjustawebapplicationand,assuch,isbuiltusingthesamesetoftools.We'llbuildours
usingPython,Flask,andSQLAlchemy
EarlierwesaidaRESTAPIallowsclientstomanipulateviaHTTP.
Prettymuch.Ifyou'resystemisbuiltusingORMmodels,yourresourcesarealmostcertainlygoingtobeyourmodels.
Webframeworksreducetheboilerplaterequiredtocreateawebapplicationbyproviding:
ofHTTPrequeststohandlerfunctionsorclassesExample:/foo=>defprocess_foo()
ofHTTPresponsestoinjectdynamicdatainpre-definedstructure
Example:<h1>Hello{{user_name}}</h1>
ThemoretimeyouspendbuildingRESTAPIswithwebframeworks,themoreyou'llnoticethesubtle(andattimes,
glaring)impedancemismatch.
URLsas toprocessingfunctions;RESTAPIstreatURLsastheaddressofaresourceorcollectionHTMLtemplating,whileRESTAPIsrarely.JSON-relatedfunctionalityfeelsbolted-on.
Imaginewe'reTwitterweeksafterlaunch.AshtonKutcherseemstobeabletouseourservice,butwhatabout
?
That'sright,we'llneedtocreateanAPI.Beinganinternetcompany,we'llbuildaRESTAPIservice.Fornow,we'llfocus
ontworesources:usertweet
Allresourcesmustbeidentifiedbyauniqueaddressatwhichtheycanbereached,theirURI.Thisrequireseachresource
containauniqueID,usuallyamonotonicallyincreasingintegerorUUID(likeaprimarykeyinadatabasetable).
OurpatternforbuildingURLswillbe/resource_name[/resource_id[/resource_attribute]]
Herewedefineourresourcesisafilecalledmodels.py:
classUser(db.Model,SerializableModel):__tablename__='user'
id=db.Column(db.Integer,primary_key=True)username=db.Column(db.String)
classTweet(db.Model,SerializableModel):__tablename__='tweet'
id=db.Column(db.Integer,primary_key=True)content=db.Column(db.String)posted_at=db.Column(db.DateTime)user_id=db.Column(db.Integer,db.ForeignKey('user.id'))user=db.relationship(User)
classSerializableModel(object):"""ASQLAlchemymodelmixinclassthatcanserializeitselfasJSON."""
defto_dict(self):"""Returndictrepresentationofclassbyiteratingoverdatabasecolumns."""value={}forcolumninself.__table__.columns:attribute=getattr(self,column.name)ifisinstance(attribute,datetime.datetime):attribute=str(attribute)value[column.name]=attributereturnvalue
Here'sthecodethathandlesretrievingasingletweetandreturningitasJSON:
frommodelsimportTweet,User
@app.route('/tweets/<int:tweet_id>',methods=['GET'])defget_tweet(tweet_id):tweet=Tweet.query.get(tweet_id)iftweetisNone:response=jsonify({'result':'error'})response.status_code=404returnresponseelse:returnjsonify({'tweet':tweet.to_dict()})
Let'scurlournewAPI(preloadedwithasingletweetanduser):
$curllocalhost:5000/tweets/1{"tweet":{"content":"Thisisawesome","id":1,"posted_at":"2014-07-0512:00:00","user_id":1}}
@app.route('/tweets/',methods=['POST'])defcreate_tweet():"""CreateanewtweetobjectbasedontheJSONdatasentintherequest."""ifnotall(('content','posted_at','user_id'inrequest.json)):response=jsonify({'result':'ERROR'})response.status_code=400#HTTP400:BADREQUESTreturnresponseelse:tweet=Tweet(content=request.json['content'],posted_at=datetime.datetime.strptime(request.json['posted_at'],'%Y-%m-%d%H:%M:%S'),user_id=request.json['user_id'])db.session.add(tweet)db.session.commit()returnjsonify(tweet.to_dict())
InRESTAPIs,agroupofresourcesiscalleda .RESTAPIsareheavilybuiltonthenotionofresourcesand
collections.Inourcase,the oftweetsisalistofalltweetsinthesystem.
ThetweetcollectionisaccessedbythefollowingURL(accordingtoourrules,describedearlier):/tweets.
@app.route('/tweets',methods=['GET'])defget_tweet_collection():"""ReturnalltweetsasJSON."""all_tweets=[]fortweetinTweet.query.all():all_tweets.append({'content':tweet.content,'posted_at':tweet.posted_at,'posted_by':tweet.user.username})
Allthecodethusfarhasbeenprettymuchboilerplate.EveryRESTAPIyouwriteinFlask(modulobusinesslogic)willlook
identical.Howcanweusethattoouradvantage?
Wehaveself-drivingcarsanddeliverydrones,whycan'twebuildRESTAPIsautomatically?
Thisallowsonetoworkatahigherlevelofabstraction.Solvetheproblemonceinageneralwayandletcodegeneration
solveeachindividualinstanceoftheproblem.
Partof
SANDBOY
ThirdpartyFlaskextensionwrittenbythedashingJeffKnupp.Defineyourmodels.Hitabutton.BAM!RESTfulAPI
servicethat .
(Thenamewillmakemoresenseinafewminutes)
GeneralizesRESTresourcehandlingintonotionofa(e.g.the"TweetService"handlesalltweet-relatedactions).classService(MethodView):"""Baseclassforallresources."""
__model__=None__db__=None
defget(self,resource_id=None):"""ReturnresponsetoHTTPGETrequest."""ifresource_idisNone:returnself._all_resources()else:resource=self._resource(resource_id)ifnotresource:raiseNotFoundExceptionreturnjsonify(resource.to_dict())
def_all_resources(self):"""ReturnallresourcesofthistypeasaJSONlist."""ifnot'page'inrequest.args:resources=self.__db__.session.query(self.__model__).all()else:resources=self.__model__.query.paginate(int(request.args['page'])).itemsreturnjsonify({'resources':[resource.to_dict()forresourceinresources]})
Here'showPOSTworks.Noticetheverify_fieldsdecoratoranduseof**request.jsonmagic...
@verify_fieldsdefpost(self):"""ReturnresponsetoHTTPPOSTrequest."""resource=self.__model__.query.filter_by(
**request.json).first()ifresource:returnself._no_content_response()instance=self.__model__(**request.json)self.__db__.session.add(instance)self.__db__.session.commit()returnself._created_response(instance.to_dict())
Wehaveourmodelsdefined.HowdowetakeadvantageofthegenericServiceclassandcreateservicesfromour
models?defregister(self,cls_list):"""RegisteraclasstobegivenaRESTAPI."""forclsincls_list:serializable_model=type(cls.__name__+'Serializable',(cls,SerializableModel),{})new_endpoint=type(cls.__name__+'Endpoint',(Service,),{'__model__':serializable_model,'__db__':self.db})view_func=new_endpoint.as_view(new_endpoint.__model__.__tablename__)self.blueprint.add_url_rule('/'+new_endpoint.__model__.__tablename__,view_func=view_func)self.blueprint.add_url_rule('/{resource}/<resource_id>'.format(resource=new_endpoint.__model__.__tablename__),view_func=view_func,methods=[
'GET','PUT','DELETE','PATCH','OPTIONS'])
TYPE
InPython,typewithoneargumentreturnsavariable'stype.Withthreearguments,
.
TYPE
serializable_model=type(cls.__name__+'Serializable',(cls,SerializableModel),{})
new_endpoint=type(cls.__name__+'Endpoint',(Service,),{'__model__':serializable_model,'__db__':self.db})
Let'splaypretendagain.Nowwe'reaIaaScompanythatletsusersbuildprivateclouds.We'llfocusontworesources:
cloudandmachine
classCloud(db.Model):__tablename__='cloud'
id=db.Column(db.Integer,primary_key=True)name=db.Column(db.String,nullable=False)description=db.Column(db.String,nullable=False)
classMachine(db.Model):__tablename__='machine'
id=db.Column(db.Integer,primary_key=True)hostname=db.Column(db.String)operating_system=db.Column(db.String)description=db.Column(db.String)cloud_id=db.Column(db.Integer,db.ForeignKey('cloud.id'))cloud=db.relationship('Cloud')is_running=db.Column(db.Boolean,default=False)
fromflaskimportFlaskfromflask.ext.sandboyimportSandboyfrommodelsimportMachine,Cloud,db
app=Flask(__name__)app.config['SQLALCHEMY_DATABASE_URI']='sqlite:///db.sqlite3'db.init_app(app)withapp.app_context():db.create_all()sandboy=Sandboy(app,db,[Machine,Cloud])app.run(debug=True)
Incaseswherewe'rebuildingaRESTAPIfromscratch,thisisprettyeasy.Butwhatif:
WehaveanexistingdatabaseWewanttocreateaRESTfulAPIforitIthas200tables
OnlydownsideofFlask-Sandboyisyouhavetodefineyourmodelclassesexplicitly.Ifyouhavealotofmodels,this
wouldbetedious.
...Idon'tdotedious
Wehaveprivatecompaniesbuildingrocketshipsandelectriccars.Whycan'twehaveatoolthatyoupointatanexistingdatabaseandhitabutton,then,BLAM!RESTfulAPIservice.
SANDMAN
,alibrarybyteenheartthrobJeffKnupp,createsaRESTfulAPIservicefor with
.
Here'showyourunsandmanagainstamysqldatabase:
$sandmanctlmysql+mysqlconnector://localhost/Chinook*Runningonhttp://0.0.0.0:8080/*Restartingwithreloader
$curl-vlocalhost:8080/artists?Name=AC/DCHTTP/1.0200OKContent-Type:application/jsonDate:Sun,06Jul201415:55:21GMTETag:"cea5dfbb05362bd56c14d0701cedb5a7"Link:</artists/1>;rel="self"
{"ArtistId":1,"Name":"AC/DC","links":[{"rel":"self","uri":"/artists/1"}],"self":"/artists/1"}
ETagsetcorrectly,allowingforcachingresponsesLinkHeadersettoletclientsdiscoverlinkstootherresourcesSearchenabledbysendinginanattributenameandvalue
Wildcardsearchingsupported
Wecancurl/andgetalistofallavailableservicesandtheirURLs.Wecanhit/<resource>/metatogetmeta-infoaboutthe
service.Example(the"artist"service):
$curl-vlocalhost:8080/artists/metaHTTP/1.0200OKContent-Length:80Content-Type:application/jsonDate:Sun,06Jul201416:04:25GMTETag:"872ea9f2c6635aa3775dc45aa6bc4975"Server:Werkzeug/0.9.6Python/2.7.6
{"Artist":{"ArtistId":"integer(11)","Name":"varchar(120)"}}
Andnowfora(probablybroken)live-demo!
"Real"RESTAPIsenableclientstousetheAPIusingonlytheinformationreturnedfromHTTPrequests.sandmantriestobeas"RESTful"aspossiblewithoutrequiringanycodefrom
theuser.
WouldbenicetobeabletovisualizeyourdatainadditiontointeractingwithitviaRESTAPI.
1. Codegeneration2. Databaseintrospection3. Lotsofmagic
sandmancamefirst.HasbeennumberonePythonprojectonGitHubmultipletimesandisdownloaded25,000timesa
month.Flask-Sandboyissandman'slittlebrother...
ThefactthattheendresultisaRESTAPIisnotespeciallyinterestingMoreimportantaretheconceptsunderpinningsandmanandFlask-Sandboy
WorkathigherlevelofabstractionSolveaproblemonceinagenericmannerReduceserrors,improvesperformance
Ingeneral:
Speakingofautomation,here'showmybookis"built"...
sandman=Flask+SQLAlchemy+LotsofGlueRequiresyouknowthecapabilitiesofyourtoolsPartoftheUNIXPhilosophy
ThebestprogrammingadviceIevergotwasto"belazy"
SandmanexistsbecauseIwastoolazytowriteboilerplateORMcodeforanexistingdatabaseFlask-SandboyexistsbecauseIwastoolazytowritethesameAPIservicesoverandoverBeinglazyforcesyoutolearnyourtoolsandmakeheavyuseofthem
Contactmeat:jeff@jeffknupp.com@jeffknupponTwitter
onthetubeshttp://www.jeffknupp.com
Recommended