Oct 12, 2015

Ansible install with openSUSE 13.2 and Mac

I start to study Ansible cause recently needs.

Ansible is python based, using SSH deploy packages to remote servers, it's like Puppet / Chef / Salt.

Online resource

I study from Oreilly - Ansible up & running
  • code example


How to install Ansible

Install Ansible

openSUSE 13.2 ( Most of linux could install ansible now )
Use zypper to query ansible package
# zypper   search   ansible
Loading repository data...
Reading installed packages...

S | Name    | Summary                                                                  | Type      
 | ansible | SSH-based configuration management, deployment, and orchestration engine | package   
 | ansible | SSH-based configuration management, deployment, and orchestration engine | srcpackage

Install Ansible
# zypper install  -y  ansible

That's all you need to do :-)

Ansible is clientless, that mean you don't need to install any packages in your remote hosts, it's the most diff with puppet and other tools.
The role is
  • The control machine
    • The one that you use to control remote machines
    • needs to have python 2.6 or later installed.  
  • To manage a server with Ansible
    • Needs to have SSH and Python 2.5 or later installed.
Most linux has python later 2.5 ( Might be python 2.7 ), that's the reason - Nothing to install

What do you need to know? ( From Oreilly book ^^ )
I think the most important point is - you could use ssh and ssh key to execute some commands in remote servers.

What do I need to know?
    • Connect to a remote machine using SSH.
    • Interact with the bash command-line shell (pipes and redirection).
    • Install packages.
    • Use the sudo command.
    • Check and set file permissions.
    • Start and stop services.
    • Set environment variables.
    • Write scripts ( Any language )
  • You don't need to know python to use Ansible unless you want to write your own module.
  • You will need to learn some YAML and Jinja2 to use Ansible.     

That's all today, I will write some Ansible module command next time.

~ enjoy it

Oct 5, 2015

Setting up Spark single node with local disk

Setting up Spark single node with local disk

OS: Ubuntu 14.04 in GENI

Please use ssh key to login Ubuntu 14.04 in GENI

If you are /bin/bash user, you might want change your shell, you could use
$sudo   chsh   -s    /bin/bash  YourUserName

Use curl command to get auto install shell script, save filename as Install.sh

$ curl   https://dl.dropboxusercontent.com/u/12787647/iCAIR/Ubuntu1404SparkInstall.sh    >   Install.sh

Use sh to run the shell script.  It will auto install spark and other packages.
$ sh  Install.sh

After install Spark in your system, you could run spark interactive shell with follow command   
$ ~/spark/bin/spark-shell

You can check the SparkUI in http://localhost:4040 like

螢幕快照 2015-08-07 下午2.59.46.png

Basic  command practice
scala> val sakanaFile = sc.textFile("README.md")
sakanaFile: org.apache.spark.rdd.RDD[String] = MapPartitionsRDD[1] at textFile at :21
scala> sakanaFile.count()
res0: Long = 98
scala> val linesWithSpark = sakanaFile.filter(line => line.contains("Spark"))
linesWithSpark: org.apache.spark.rdd.RDD[String] = MapPartitionsRDD[2] at filter at :23
scala> linesWithSpark.count()
res1: Long = 19
scala> linesWithSpark.collect()

scala> linesWithSpark.collect.foreach(println)