View
230
Download
0
Category
Preview:
Citation preview
Msrks’s Dev Notes DocumentationRelease 1.0
Masahiro Rikiso
May 10, 2017
Scientific Python
1 Deep Learning 1
2 Mosaik 3
3 Python 7
4 Docker 9
5 Heroku 13
6 Vagrant 15
7 Bootstrap 17
8 D3JS 21
9 JQuery 23
10 Atom 25
11 Linux 27
12 Local Server 29
13 Raspberry Pi 31
14 Read the Docs 33
i
ii
CHAPTER 1
Deep Learning
• Deep Learning
• Personal Page
1
Msrks’s Dev Notes Documentation, Release 1.0
2 Chapter 1. Deep Learning
CHAPTER 2
Mosaik
mosaik is a simulator manager!
resource
• https://mosaik.offis.de
• http://mosaik.readthedocs.io/en/latest/index.html
installation
mosaik:
$ cd mosaik$ pyenv virtualenv 3.5.0 mosaik35$ pyenv local mosaik35$ pip install mosaik
mosaik-demo:
$ brew install hg gcc hdf5$ pip install numpy scipy h5py$ hg clone https://bitbucket.org/mosaik/mosaik-demo$ cd mosaik-demo$ pip install -r requirement.txt$ python demo.py
Concept
4 main component
3
Msrks’s Dev Notes Documentation, Release 1.0
• mosaik Sim API(communicate between simulators and mosaik)
• Scenario API
• Simulator Manager(handle simulator processes)
• simulator(schedule)
4 module
• mosaik-core
• mosaik-componets(example simulator)
• mosaik-tools(analysis tools)
• Odysseus(big-data-analysis)
How to use
1. create simulation wrapper
1. define Meta Data:
META = {'models': {
'ExampleModel': {'public': True,'params': ['init_val'],'attrs': ['delta', 'val'],
},},
}
2. define Simulator Wrapper:
class ExampleSim(mosaik_api.Simulator):def __init__(self):
super().__init__(META)self.simulator = simulator.Simulator()self.eid_prefix = 'Model_'self.entities = {} # Maps EIDs to model
# additional simulator initializaitondef init(self, sid):
return self.meta
# model instance initializaitondef create(self, num, model, init_val):
entities = []## model creation#entities.append({'eid': eid, 'type': model})return entities
def step(self, time, inputs):## peform simulation step
4 Chapter 2. Mosaik
Msrks’s Dev Notes Documentation, Release 1.0
#return nexttime
def get_data(self, query):## construct returndata#return data
Note: simulator wrapper is subclass of mosaiki_api.Simulator
3. making it executable:
def main():return mosaik_api.start_simulation(ExampleSim())
if __name__ == '__main__':main()
2.4. How to use 5
Msrks’s Dev Notes Documentation, Release 1.0
6 Chapter 2. Mosaik
CHAPTER 3
Python
Python
pyenv
upgrade:
$ brew update$ brew upgrade pyenv
os:
import osfor f in os.listdir('txt/'):# do something
import osif not os.path.exists(FOLDER):
os.makedirs(FOLDER)
pandas
thousands:
df = pd.read_csv('txt/'+f, sep='\t',names=['delay','count'],thousands=',',dtype={'delay':str, 'count':int})
7
Msrks’s Dev Notes Documentation, Release 1.0
save csv:
df.to-csv("filename", index=False)
matplotlib, seaborn
rc:
matplotlib.rc('font', size=12)matplotlib.rc('axes', labelsize=16, titlesize=16)matplotlib.rc('xtick', labelsize=16)matplotlib.rc('ytick', labelsize=16)plt.rc('text', usetex=True)#sns.set(font_scale=1.4)
savefig:
fig.savefig("hoge.png")
8 Chapter 3. Python
CHAPTER 4
Docker
Docker
:
$ docker images$ docker run --name NAME -i -t (-d)\ # -d:
-v HOST-DIR:CONTAINER-DIR \-p HOST-PORT:CONTAINER-PORT \IMAGE /bin/bash
$ docker rmi IMAGE$ docker ps (-a)$ docker start CONTAINER$ docker stop CONTAINER$ docker attach CONTAINER #$ docker rm CONTAINER
Build:
$ mkdir mydockerbuild$ cd mydockerbuild$ touch Dockerfile#https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/$ atom Dockerfile
.. FROM docker/whalesay:latest
.. MAINTAINER Masahiro Rikiso <mshrrks@gmail.com>
.. RUN apt-get -y update && apt-get install -y fortunes
.. ADD ..
.. ENV ..
.. EXPOSE 80
.. CMD /usr/games/fortune -a | cowsay$ docker build -t <image-name> .
Commit:
9
Msrks’s Dev Notes Documentation, Release 1.0
$ docker commit <container-id> <image-name>
Tag & Push:
$ docker tag <image-id> msrks/<image-name>$ docker login --username=msrks --email=mshrrks@gmail.com$ docker push msrks/<image-name>
:
$ docker save IMAGE > i-name.tar$ docker load < i-name.tar$ docker export CONTAINER > c-name.tar
:
$ docker cp CONTAINER:filename ./
:
$ docker diff CONTAINER
gitbucket
$ docker run --name gitbucket-8001 -d -p 8001:8080 takezoe/gitbucket
tensorflow:
$ docker run --name jupyter-8002 -d -p 8002:8888 gcr.io/tensorflow/tensorflow
caffe:
$ docker run --name jupyter-8003 -d -t -p 8003:8888 msrks/caffe-demo \/usr/local/bin/jupyter notebook --notebook-dir=/opt/caffe/examples/ --ip=0.0.0.0
caffe-webdemo:
$ docker run --name webdemo-8004 -d -t -p 8004:5000 msrks/caffe-demo \/usr/bin/python /opt/caffe/examples/web_demo/app.py
sphinx-latexpdf:
$ sudo docker run --name "sphinx" -v=/HOST/doc:/docs \-it higebu/sphinx-latexpdf /bin/bash(docker) # pip install -U sphinx_rtd_theme --proxy=158.202.41.115:8080(docker) # cd docs(docker) # make latexpdfja
10 Chapter 4. Docker
Msrks’s Dev Notes Documentation, Release 1.0
:
$ docker ps -a
:
$ docker start <container-id>
4.2. 11
Msrks’s Dev Notes Documentation, Release 1.0
12 Chapter 4. Docker
CHAPTER 5
Heroku
resource:
• https://devcenter.heroku.com/articles/getting-started-with-python#introduction
• https://github.com/heroku/python-getting-started
• http://kennmyers.github.io/tutorial/2016/03/11/getting-flask-on-heroku.html
Basic
installation
1. create account on heroku
2. download and instal heroku toolbelt
3. heroku login
deploy python app:
$ mkdir heroku-app$ cd heroku-app$ pyenv virtualenv 2.7.10 venv$ pyenv local venv$ echo .python-version > .gitignore$ git init
$ pip install gunicorn$ pip freeze > requirements.txt$ echo python-2.7.10 > runtime.txt$ echo web: gunicorn app:app --log-file=- > Procfile
#$ heroku local web
$ heroku create <app-name>
13
Msrks’s Dev Notes Documentation, Release 1.0
$ heroku buildpacks:set heroku/python
$ git add -A$ git commit -m "initial commit"$ git push heroku master$ heroku ps:scale web=1$ heroku open
log:
$ heroku logs --tail
14 Chapter 5. Heroku
CHAPTER 6
Vagrant
resource:
https://www.vagrantup.com
boxes:
https://atlas.hashicorp.com/boxes/search
vagrantfile
example:
# -*- mode: ruby -*-# vi: set ft=ruby :
Vagrant.configure("2") do |config|config.vm.box = "ubuntu/trusty64"config.vm.network "forwarded_port", guest: 80, host: 8080config.vm.network "private_network", ip: "192.168.33.10"# config.vm.network "public_network"# config.vm.synced_folder "../data", "/vagrant_data"
# config.vm.provider "virtualbox" do |vb|# # Display the VirtualBox GUI when booting the machine# vb.gui = true## # Customize the amount of memory on the VM:# vb.memory = "1024"# end
# config.vm.provision "shell", inline: <<-SHELL# apt-get update# apt-get install -y apache2
15
Msrks’s Dev Notes Documentation, Release 1.0
# SHELLend
16 Chapter 6. Vagrant
CHAPTER 7
Bootstrap
<!doctype html><html lang="ja"><head><meta charset="utf-8"><title>test</title><meta name="viewport" content="width=device-width, initial-scale=1.0"><link href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/css/bootstrap.min.css"
→˓rel="stylesheet" media="screen"><script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.2/jquery.min.js"></
→˓script><script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js"></
→˓script><style>body {
font-family: Verdana, Arial, sans-serif;-webkit-font-smoothing: antialiased;
}h1,h2,h3 {
font-family: Times, serif;line-height: 1.5em;border-bottom: 1px solid #ccc;
}</style>
</head><body></body></html>
17
Msrks’s Dev Notes Documentation, Release 1.0
body
• container
• col-md-X (row): grid system
• header navbar
:
<div class="container"><!-- navbar --><div class="row"><nav class="navbar navbar-default navbar-fixed-top"><div class="navbar-header"><a href="" class="navbar-brand">Home</a>
</div><ul class="nav navbar-nav"><li class="active"><a href="">menu1</a></li><li><a href="">menu2</a></li><li><a href="">menu3</a></li><li><a href="">menu4</a></li>
</ul></nav>
</div><!-- content --><div class="row" style="padding:60px 0 0 0"><div class="col-md-3" align="center">left</div><div class="col-md-9" align="center">center</div>
</div><!-- footer --><hr><p>© Masahiro Rikiso</p>
</div>
table:
<table class="table table-striped table-boarderd table-hover"><thead><tr><th>ID</th><th>Score</th></tr>
</thead><tbody><tr><tb>python</tb><tb>100</tb></tr>
</tbody></table>
Note: class=”table table-striped table-boarderd table-hover”
form:
<form><div class="form-group">
18 Chapter 7. Bootstrap
Msrks’s Dev Notes Documentation, Release 1.0
<label class="control-label" for="email">Email</label><input type="text" id="email" class="form-control" placeholder="email">
</div><div class="form-group"><input type="submit" value="submit" class="btn btn-primary">
</div></form>
Note: class=”form-group” input text “form-control” input submit “btn btn-primary”
7.3. 19
Msrks’s Dev Notes Documentation, Release 1.0
20 Chapter 7. Bootstrap
CHAPTER 8
D3JS
Setting
d3js :
<script src="http://d3js.org/d3.v3.min.js" charset="utf-8"></script>
Tips
DOM:
select, selectAll
:
text, attr, style, append, remove
:
data, enter, exit
:
<body><p>1</p><p>2</p>
<script>var dataset = [1,2,3]var p = d3.select("body").selectAll("p")var update = p.data(dataset)var enter = update.enter().append(p)//var exit= update.exit()
21
Msrks’s Dev Notes Documentation, Release 1.0
</script></body>
svg:
<body><script>var dataset = [1,2,3,4,5]var svg = d3.select("body").append("svg")
.attr("width", 500)
.attr("height", 200);</script></body>
animation:
transition, delay, duration, ease,each("start",function(){}), each("end",function(){})
event:
on("mouseover", function(){})on("mouseout", function(){})on("mousedown", function(){})on("mouseup", function(){})on("click", function(){})
22 Chapter 8. D3JS
CHAPTER 9
JQuery
id=classifyfiledisabled=trueid=imagefileformsubmit:
<head><script type="text/javascript">$(document).ready(function(){$('#classifyfile').attr('disabled',true);$('#imagefile').change(
function(){if ($(this).val()){
$('#formupload').submit();}
});
});
</script></head><body><form id="formupload" class="form-inline" role="form" action="classify_upload"
→˓method="post" enctype="multipart/form-data"><div class="form-group"><label for="imagefile">Or upload an image:</label><input type="file" name="imagefile" id="imagefile">
</div><!--<input type="submit" class="btn btn-primary" value="Classify File" id=
→˓"classifyfile"></input>--></form>
</body>
23
Msrks’s Dev Notes Documentation, Release 1.0
24 Chapter 9. JQuery
CHAPTER 10
Atom
Proxy
Linux
@160506:
$ apm config set http-proxy http://<ip>:<port>$ apm config set https-proxy https://<ip>:<port>$ apm config set strict-ssl false$ export ATOM_NODE_URL=http://gh-contractor-zcbenz.s3.amazonaws.com/atom-shell/dist
Windows
@160623:
# 1) $HOME/.atom/.apmrchttp-proxy=http://<ip>:<port>https-proxy=https://<ip>:<port>strict-ssl=false# 2)$ setx ATOM_NODE_URL http://gh-contractor-zcbenz.s3.amazonaws.com/atom-shell/dist /M# 3)$ apm config set http-proxy http://<ip>:<port>$ apm config set https-proxy http://<ip>:<port>$ apm config set strict-ssl false
25
Msrks’s Dev Notes Documentation, Release 1.0
26 Chapter 10. Atom
CHAPTER 11
Linux
ip:
$ sudo ifconfig eth0 inet 10.0.0.100 netmask 255.0.0.0 up$ sudo route add default gw 10.0.0.1
SNMP server
netsnmp:
$ sudo apt-get install snmp sniped$ snmpwalk -v 2c -c public <ip-addr> <mib-dir>
DHCP server
isc-dhcp-server:
$ sudo apt-get install isc-dhcp-server$ sudo leafpad /etc/dhcp/dhcp.conf
# dhcp-relaysubnet 172.16.0.0 netmask 255.255.255.0 {}subnet 192.168.0.0 netmask 255.255.255.0 {
range 192.168.0.2 192.168.0.250;option routers 192.168.0.1;option subnet-mask 255.255.255.0# dns setting##
27
Msrks’s Dev Notes Documentation, Release 1.0
}$ sudo leafpad /etc/default/isc-dhcp-server
# INTERFACE = "eth0"$ sudo touch /var/lib/dhcp/dhcpd.leases$ sudo systemctl stop isc-dhcp-server.service$ sudo systemctl start isc-dhcp-server.service$ sudo systemctl enable isc-dhcp-server.service
Packet Capture
wireshark:
$ sudo apt-get install wireshark$ sudo groupadd wireshark$ sudo usermod -a -G wireshark pi$ sudo chgrp wireshark /usr/bin/dumpcap$ sudo chmod 750 /usr/bin/dumpcap$ sudo raspi-config$ sudo setcap cap_net_raw.cap_net_admin=eip /usr/bin/dumpcap$ sudo chmod 4711 'which dumpcap'$ sudo chmod 4711 /usr/bin/dumpcap
28 Chapter 11. Linux
CHAPTER 12
Local Server
python2(port=8000):
$ python -m SimpleHTTPServer
python3(port=8000):
$ python -m http.server
python2:
$ wget https://gist.github.com/msrks/18125b9a57e69e0309c8f743c87904d6/raw/→˓ff5a398ce7d828137869f91796446300b64c9941/server2.py$ python server2.py USER:PASS
python3:
$ wget https://gist.github.com/msrks/3fe0e3287f548382f442a9c79d5f5ec9/raw/→˓3416832d494feec0dda1b05b0190986f59878153/server3.py$ python server3.py USER:PASS
29
Msrks’s Dev Notes Documentation, Release 1.0
30 Chapter 12. Local Server
CHAPTER 13
Raspberry Pi
OS Installation @ Mac OS X
@160501
1. RPI official RASPBIAN JESSIE()
2. zip:
$ cd Downloads$ tar zxvf 2016-05-27-raspbian-jessie.zip
3. SD(Mac book /dev/disk3 ):
$ diskutil list
4. SD:
$ sudo diskutil eraseDisk FAT32 RPI /dev/disk3Started erase on disk3Unmounting diskCreating the partition map...Finished erase on disk3
5. SD:
$ sudo diskutil unmountDisk /dev/disk3Unmount of all volumes on disk3 was successful
6. SD:
$ sudo dd bs=1m if=2016-05-27-raspbian-jessie.img of=/dev/disk33125+0 records in
31
Msrks’s Dev Notes Documentation, Release 1.0
3125+0 records out3276800000 bytes transferred in 2784.822319 secs (1176664 bytes/sec)
7. RPISD:
pi@raspberrypi ~ $ sudo raspi-config
ip
/etc/network/interfaces dhcp‘‘/etc/dhcpcd.conf‘‘ :
$ sudo leafpad /etc/dhcpcd.conf$ sudo /etc/init.d/dhcpcd reload
32 Chapter 13. Raspberry Pi
CHAPTER 14
Read the Docs
githubrepoclone:
$ git clone <github.repo>$ cd <project>
build-dir:
$ touch .gitignore$ echo docs/_build > .gitignore
doc:
$ mkdir docs$ cd docs$ sphinx-quickstart
conf.py:
# defaulthtml_theme = "default"
githubpush:
$ cd ..$ git add -A$ git commit -m "[add] docs"$ git push
Read the Docs <project>importbuild
33
Recommended