Neo4j Spark connector Tutorial

Neo4j having python drivers could be availed from this  location
Neo4j spark connector available at location
It is having drivers for java and scala but not for python

Below are the steps to setup Neo4j- spark scala connector application

Setup:
1. neo4j spark connector
Connector jar could be downloaded from the location
2. scala
download rpm from location
and install using rpm -i
3. sbt
Download sbt to /opt from location
update /etc/profile with export PATH=$PATH:/opt/sbt/bin

Code:
Write scala code or checkout example from github.com location
Update neo4j credentials

Build & Run:
Follow steps in the Readme to build & run the application 

MIcroservice Case study

Microservices could be implemented in several way. In this case below is the stack used.

• Django
• Flask
• Node.js
• mysql
• mongo
• docker
• kubernetes
• ubuntu 

In this usecase API searches keyword "search_term" in the available public repositories and displays top 5 active repositories

There are 3 app services here i.e., Django (DAPP) service, FLASK(FAPP) service, NODE.JS(Nodeapp) service

There are 2 database services Mysql(ms-mysql)service and Mongo(ms-mongo)service.

All five docker images are available at location Docker Hub

Preparing the environment

Install docker and kubernetes as mentioned in the location 

Checkout and start kubernetes files from github location

Starting and Stopping services

Start the kubernetes services/pods/deployments with below steps with the files located in dhub

sh ./start0.sh

sh ./start.sh

For stopping kubernetes services/pods/deployments use below step

sh ./stop.sh

Consuming APIs

DAPP consumes FAPP, MySql services and exposes API

http://<HOST-IP>:8000/search/?search_term=flask

FAPP consumes Mongo service and exposed API

http://<HOST-IP>:9000/api/v1/githubsearch/?search_term=kubernetes&employee=nexi_user3

Nodeapp consumes Mongo service

http://<HOST-IP>:8081/

Deployin django in nginx & uwsgi environment

1. Install uwsgi , nginx 
apt-get install uwsgi, nginx

2. Clone django helloworld program
git clone https://github.com/Jadatravu/Tutorials/

3. Copy the helloworld program into /opt folder
cp -r Tutorials/dj_nx_wsgi  /opt

sudo chown -R www-data:www-data /opt/dj_nx_wsgi

4. Create a python virtual environment and install django
cd /opt
virtualenv env
source /opt/env/bin/activate
pip install django

5. Test uwsgi
(env) osboxes@ubuntu:/opt$ uwsgi --http  :8001 --wsgi-file /opt/dj_ng_wsgi/dj_ng_wsgi/wsgi.py  --virtualenv /opt/env --chdir /opt/dj_ng_wsgi/

6. Access from browser
http://192.168.91.129:8001/hello/

7. Configure nginx
copy nginx.conf from dj_nj_wsgi folder to /etc/nginx/sites-available
make soft link
ln -s /etc/nginx/sites-available/nginx.conf /etc/nginx/sites-enabled/nginx.conf

8.restart nginx
service nginx restart

9. start wsgi
(env) osboxes@ubuntu:/opt$ uwsgi --socket :8001 --wsgi-file /opt/dj_ng_wsgi/dj_ng_wsgi/wsgi.py  --virtualenv /opt/env --chdir /opt/dj_ng_wsgi/

10.from browser access the url.
http://192.168.91.129:8000/hello/

pylint, python logger

pylint --disable=R,C,W,I,E,F test.py --msg-template='{msg_id}:{line:3d},{column}: {obj}: {msg}' --disable=import-error --rcfile ./to-path/.pylintrc

import logging

import sys

root = logging.getLogger()

root.setLevel(logging.DEBUG)

fo= open("/Users/sadatravu/Surya/py_scripts/text.log", "w")

ch= logging.StreamHandler(fo)

#ch = logging.StreamHandler(sys.stdout) # this is for logging into stdout

ch.setLevel(logging.DEBUG)

formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')

ch.setFormatter(formatter)

root.addHandler(ch)

def main():

    for i in range(10):

       logging.debug("message [%d]"%i)

main()

SPA using Angular

Single Page Application using Angular

Video

knockout-vs-backbone-vs-angular

• Angular and knockout comes with data binding
• Backbone is also quite easy to grasp.
• Angular is very powerful and deals with complex models

Original blog

DJANGO FAQ

1. Django project files significance

a. manage.py => enables the django project settings path to the environment
b. settings.py => this file contains the settings of django project like debug mode, database settings, templates path, static_url path etc.,
c. urls.py this file is the controller file which maps the web url to the respective view function/class
d. wsgi.py file => this file will have the environment settings like virtual environment site-packages path, django project path. path of this wsgi.py should be included in the apache configuration file apache2.conf
e. models.py => this file will have the information models and members of the models
f. views.py => this file will have the views functions/classes

2. Django commands

django-admin start-project my_pro => Creates django project
django-admin startapp myapp => creates django application inside the django project

python ./manage.py runserver 127.0.0.1:8000 => runs the development server

python ./manage.py makemigrations => creates the database migration scripts

python ./manage.py migrate => applies the migrations

python ./manage.py inspectdb > model.py  => Creates the models wrt to the already existing database

python ./manage.py collectstatic => collects all the static files in the django project

python ./manage.py dbshell  => Opens the database shell

python ./manage.py shell => opens the django/python shell