sillycat
- 浏览: 2489669 次
- 性别:
- 来自: 成都
-
最新评论
-
nation：
你好，在部署Mesos+Spark的运行环境时，出现一个现象， ...
Spark(4)Deal with Mesos -
sillycat：
AMAZON Relatedhttps://www.godad ...
AMAZON API Gateway(2)Client Side SSL with NGINX -
sillycat：
sudo usermod -aG docker ec2-use ...
Docker and VirtualBox(1)Set up Shared Disk for Virtual Box -
sillycat：
Every Half an Hour30 * * * * /u ...
Build Home NAS(3)Data Redundancy -
sillycat：
3 List the Cron Job I Have>c ...
Build Home NAS(3)Data Redundancy
文章列表
Supervisord Trouble Shooting on CentOS7
Recently, my colleague enlarge the memory of our cloud machine, after that, supervisord is not running.
Event I re-install that, it can not start. No error, no exception, nothing useful from logging.
Finally, I checked the systemctl configuration, I found the ...
Python Library 2019(1)requests and aiohttp
Try AIOHttp
> mkdir aiohttp
> cd aiohttp/
Check Version
> python -V
Python 3.7.5
> pip -V
pip 19.3.1 from /Users/hluo/.pyenv/versions/3.7.5/lib/python3.7/site-packages/pip (python 3.7)
> pip install aiohttp
> pip install cchardet
> p ...
Hadoop Docker 2019 Version 3.2.1
I try to set up a HDFS in Docker, which can be running on one server to provide DFS. That is it. The files there can be easily share with multiple machines.
Exception:
> systemctl start sshd
Failed to get D-Bus connection: Operation not permitted
Solution:
I can ...
Nginx and Proxy 2019(1)Nginx Enable Lua and Parse JSON
Set Up Nginx
> wget http://nginx.org/download/nginx-1.16.1.tar.gz
> tar zxvf nginx-1.16.1.tar.gz
> ./configure --prefix=/home/carl/tool/nginx-1.16.1
> make
> make install
> sudo ln -s /home/carl/tool/nginx-1.16.1 /opt/nginx-1. ...
Monitor and Alarm 2019(1)Prometheus Grafana Alertmanager
Find the download from here
https://prometheus.io/download/
I choose the Operating System Linux, Architecture amd64
> wget https://github.com/prometheus/prometheus/releases/download/v2.14.0/prometheus-2.14.0.linux-amd64.tar.gz
> tar zxvf ...
MongoDB 2019(3)Security and Auth
Started the mongo cluster first
Connect to it
> mongo -host rancher-worker2 -port 27017
> rs.slaveOk();
> db.users.find();
Check DB
> show dbs
admin 0.000GB
config 0.000GB
local 0.000GB
test 0.000GB
Switch to use admin
> use admin
switched to ...
MongoDB 2019(1)
Follow this https://docs.mongodb.com/manual/tutorial/install-mongodb-on-red-hat-tarball/
Prepare the package for mongodb
> sudo yum install libcurl openssl
Select Version 4.2.1 current release version, OS RHEL 7.0 Linux 64-bit x64 for my CentOS7, Package TGZ
> wget https://fas ...
Data Solution 2019(13)Docker Zeppelin Notebook and Memory Configuration
On my MAC, I run into this error when I build my docker image
Disk Requirements:
At least 187MB more space needed on the / filesystem.
I check my disk space, I do have disk on MAC. So maybe it caused by I build too many dock ...
Data Solution 2019(12)Flink Processing Data
Master and Slaves Mode
> java -version
java version "1.8.0_221"
Start from here
https://ci.apache.org/projects/flink/flink-docs-release-1.9/getting-started/tutorials/local_setup.html
https://juejin.im/post/5d6610c65188257573636a86
> wget ...
Data Solution 2019(10)Spark Cluster Solution with Zeppelin
Spark Single Cluster
https://spark.apache.org/docs/latest/spark-standalone.html
Mesos Cluster
https://spark.apache.org/docs/latest/running-on-mesos.html
Hadoop2 YARN
https://spark.apache.org/docs/latest/running-on-yarn.html
K8S
https://spark ...
Data Solution 2019(9)CentOS Installation
Join two DataFrame and get the count
import org.apache.spark.sql.functions._
val addressCountDF = addressesRawDF.groupBy("user_id").agg(count("user_id").as("times"))
val userWithCountDF = usersRawDF.join(
addressCountDF,
user ...
NodeJS Installation 2019
Install 10.16.3 on MAC
> wget https://nodejs.org/dist/v10.16.3/node-v10.16.3.tar.gz
> tar zxvf node-v10.16.3.tar.gz
> cd node-v10.16.3
> ./configure --prefix=/Users/hluo/tool/node-v10.16.3
Get some error about PYENV and Python2.7
> pyenv shell 3.8.0 2.7.10
E ...
Monitor Tool 2019(2)Monit on Multiple Instances and Email Alerts
We can visit the page URL on all the instances
http://rancher-home:2812/
Get some XML feed back as follow:
http://rancher-home:2812/_status?format=xml
Run the command as sudo/root
cat haproxy
check process haproxy with pidfile /opt/ ...
Monitor Tool 2019(1)Monit Installation and Usage
On CentOS 7
> sudo yum install monit
Check version
> monit -V
This is Monit version 5.25.1
Built with ssl, with ipv6, with compression, with pam and with large files
Copyright (C) 2001-2017 Tildeslash Ltd. All Rights Reserved.
Some useful comma ...
Ansible 2019(1)Introduction and Installation on Ubuntu and CentOS
Ansible is using SSH, it can execute the shell commands.
Install Ansible on Ubuntu
> sudo apt-add-repository -y ppa:ansible/ansible
> sudo apt-get update
> sudo apt-get install ansible
> ansible --version
ansible 2.8.5
c ...