Fork and Exec Using on JAVA
Recently I am reading the codes which write by my others. I think it will cause us problem in the future. It is like a small bomb on our server.
The system is using scala codes as follow to execute shell script
Logger.debug(String.format("Executing script: %s %s %s", "/bin/sh", shellScript, date))
val process = Runtime.getRuntime().exec(Array("/bin/sh", shellScript, date))
process.waitFor()
Logger.debug("Script execution finished, process output.")
val buf = new StringBuffer
val infos = Source.fromInputStream(process.getInputStream())(Codec.UTF8).getLines
val errors = Source.fromInputStream(process.getErrorStream())(Codec.UTF8).getLines
if (!infos.isEmpty) {
buf.append("Info:<br/>")
infos.foreach(buf.append(" ").append(_).append("<br/>"))
}
if (!errors.isEmpty) {
buf.append("Error:<br/>")
errors.foreach(buf.append(" ").append(_).append("<br/>"))
}
buf.toString()
Then it will call java.lang.Runtime.java, I am reading the codes from oracle jdk1.7 from my understanding.
public Process exec(String[] cmdarray, String[] envp, File dir)
throws IOException {
return new ProcessBuilder(cmdarray)
.environment(envp)
.directory(dir)
.start();
}
It will call the method java.lang.ProcessBuilder.java
return ProcessImpl.start(cmdarray,
environment,
dir,
redirects,
redirectErrorStream);
Open the file java.lang.ProcessImpl.java
return new UNIXProcess
(toCString(cmdarray[0]),
argBlock, args.length,
envBlock, envc[0],
toCString(dir),
std_fds,
redirectErrorStream);
Then Open the file is java.lang.UNIXProcess.java, it is the native codes.
private native int forkAndExec(int mode, byte[] helperpath,
byte[] prog,
byte[] argBlock, int argc,
byte[] envBlock, int envc,
byte[] dir,
int[] fds,
boolean redirectErrorStream)
throws IOException;
Then I check the man book on linux system from here
http://linux.die.net/man/2/fork
From the document, it is saying that
Under Linux, fork() is implemented using copy-on-write pages, so the only penalty that it incurs is the time and memory required to duplicate the parent's page tables, and to create a unique task structure for the child.
That means if the parent application is running in 4G memory, this sub process will use 4G too.
Actually our situation is worse
-Xmx6144m -Xmn2048m -XX:PermSize=192m -XX:MaxPermSize=192m
How to check the free memory on my machine
http://blog.scoutapp.com/articles/2010/10/06/determining-free-memory-on-linux
free -m total used free shared buffers cached Mem: 8008 7327 680 0 153 690 -/+ buffers/cache: 6483 1524 Swap: 0 0 0
The actually free memory will be Free(680 MB) + Buffers ( 153 MB) + cached (690 MB) = 1523 MB
On that shell script, we are using shell script only doing the curl(actually it is wget), zip (compress file), ftp related operations. So we have 2 options.
1. Do not use Shell Script
Using Open Source FTP Client http://commons.apache.org/proper/commons-net/
I read the source codes, it is using socket to deal with ftp protocol, there is no shell script there.
http://svn.apache.org/repos/asf/commons/proper/net/trunk/src/main/java/org/apache/commons/net/ftp/FTPClient.java
Using Open Source API to compress the file
http://commons.apache.org/proper/commons-compress/
2. Using a small operation provide restAPI to execute shell command
A standalone app with little memory.
References:
http://blog.csdn.net/vernonzheng/article/details/8644936
http://linux.die.net/man/2/fork
ftp client
http://commons.apache.org/proper/commons-net/
compress
http://commons.apache.org/proper/commons-compress/
- 浏览: 2486451 次
- 性别:
- 来自: 成都
文章分类
最新评论
-
nation:
你好,在部署Mesos+Spark的运行环境时,出现一个现象, ...
Spark(4)Deal with Mesos -
sillycat:
AMAZON Relatedhttps://www.godad ...
AMAZON API Gateway(2)Client Side SSL with NGINX -
sillycat:
sudo usermod -aG docker ec2-use ...
Docker and VirtualBox(1)Set up Shared Disk for Virtual Box -
sillycat:
Every Half an Hour30 * * * * /u ...
Build Home NAS(3)Data Redundancy -
sillycat:
3 List the Cron Job I Have>c ...
Build Home NAS(3)Data Redundancy
发表评论
-
Update Site will come soon
2021-06-02 04:10 1609I am still keep notes my tech n ... -
Stop Update Here
2020-04-28 09:00 260I will stop update here, and mo ... -
NodeJS12 and Zlib
2020-04-01 07:44 430NodeJS12 and Zlib It works as ... -
Docker Swarm 2020(2)Docker Swarm and Portainer
2020-03-31 23:18 310Docker Swarm 2020(2)Docker Swar ... -
Docker Swarm 2020(1)Simply Install and Use Swarm
2020-03-31 07:58 321Docker Swarm 2020(1)Simply Inst ... -
Traefik 2020(1)Introduction and Installation
2020-03-29 13:52 291Traefik 2020(1)Introduction and ... -
Portainer 2020(4)Deploy Nginx and Others
2020-03-20 12:06 378Portainer 2020(4)Deploy Nginx a ... -
Private Registry 2020(1)No auth in registry Nginx AUTH for UI
2020-03-18 00:56 373Private Registry 2020(1)No auth ... -
Docker Compose 2020(1)Installation and Basic
2020-03-15 08:10 325Docker Compose 2020(1)Installat ... -
VPN Server 2020(2)Docker on CentOS in Ubuntu
2020-03-02 08:04 397VPN Server 2020(2)Docker on Cen ... -
Buffer in NodeJS 12 and NodeJS 8
2020-02-25 06:43 334Buffer in NodeJS 12 and NodeJS ... -
NodeJS ENV Similar to JENV and PyENV
2020-02-25 05:14 415NodeJS ENV Similar to JENV and ... -
Prometheus HA 2020(3)AlertManager Cluster
2020-02-24 01:47 359Prometheus HA 2020(3)AlertManag ... -
Serverless with NodeJS and TencentCloud 2020(5)CRON and Settings
2020-02-24 01:46 291Serverless with NodeJS and Tenc ... -
GraphQL 2019(3)Connect to MySQL
2020-02-24 01:48 207GraphQL 2019(3)Connect to MySQL ... -
GraphQL 2019(2)GraphQL and Deploy to Tencent Cloud
2020-02-24 01:48 390GraphQL 2019(2)GraphQL and Depl ... -
GraphQL 2019(1)Apollo Basic
2020-02-19 01:36 274GraphQL 2019(1)Apollo Basic Cl ... -
Serverless with NodeJS and TencentCloud 2020(4)Multiple Handlers and Running wit
2020-02-19 01:19 263Serverless with NodeJS and Tenc ... -
Serverless with NodeJS and TencentCloud 2020(3)Build Tree and Traverse Tree
2020-02-19 01:19 259Serverless with NodeJS and Tenc ... -
Serverless with NodeJS and TencentCloud 2020(2)Trigger SCF in SCF
2020-02-19 01:18 250Serverless with NodeJS and Tenc ...
相关推荐
译文:Fork and Join: Java Can Excel at Painless Parallel Programming Too!(Fork and Join Java也可以轻松地编写并发程序)
fork函数是Unix系统最杰出的成就之一,它是七十年代UNIX早期的开发者经过长期在理论和实践上的艰苦探索后取得的成果,一方面,它使操作系统在进程管理上付出了最小的代价,另一方面,又为程序员提供了一个简洁明了的...
Unix课程作业。 使用fork(), exec(), dup2(), pipe() ,open()系统调用完成与下列shell命令等价的功能。 grep –v usr < /etc/passwd | wc –l > result.txt
Java并发网文归并总结。Fork and join,比较详细
fork和exec系列调用前后,进程在内存的“数据段”,“堆栈段”和“代码段”有什么不同?读了之后,你就会知道了,
使用fork(),exec(),dup2(), pipe(),open()系统调用完成与下列shell命令等价的功能:grep -v usr | wc -l > result.txt
1、fork 函数介绍 2、典型使用 fork 的代码片段: 3、示例 1、exec 系列函数介绍 2、头文件: 3、示例
java Fork Join框架及使用,java自带的多线程框架,来处理多线程的问题
系统调用fork( ), exec系列函数,exit( ), wait( )的功能和实现过程 使用系统调用fork( )创建两个子进程。当此程序运行时,在系统中有一个父进程和两个子进程活动。让每一个进程在屏幕上显示一个字符:父进程显示...
给 Fork me on GitHub 换个风格.zip,A fresher "Fork me on GitHub" callout.
该事例程序属多进程调用小程序,采用了系统调用函数fork()进行子进程的调用。在该文档中介绍了fork()调用的相关细节及注意点以及父子进程段的关系。并将程序运行后输出到屏幕结果与输出重定向到文件后的结果进行比较...
主要介绍了Python中的进程分支fork和exec详解,本文用实例讲解fork()的使用,并讲解了exec相关的8个方法等内容,需要的朋友可以参考下
这篇文章主要介绍了Python中的进程分支fork和exec详解,本文用实例讲解fork()的使用,并讲解了exec相关的8个方法等内容,需要的朋友可以参考下 在python中,任务并发一种方式是通过进程分支来实现的.在linux系统在,通过...
This concise book empowers all Java developers to master the complexity of the Java thread APIs and concurrency utilities. This knowledge aids the Java developer in writing correct and complex ...
Created by Yuntao Qiang, for OS Class. Pls feel free to use this resource
使用linux来fork程序 "Fork",除了它是一個當你不停地敲入后看起來非常奇怪的單詞以外,通常是指 Unix 產生新進程的方式。由于系統調用的用法將會在其他 IPC 的文檔中出現,本文只是一個快速的,不太精确的 fork() ...
Java7之forkjoin简介_动力节点Java学院整理,动力节点口口相传的Java黄埔军校
Oracle has two products that implement Java Platform Standard Edition (Java SE) 7: Java SE Development Kit (JDK) 7 and Java SE Runtime Environment (JRE) 7. JDK 7 is a superset of JRE 7, and contains ...
fork/join框架是ExecutorService接口的一个实现,可以帮助开发人员充分利用多核处理器的优势,编写出并行执行的程序,提高应用程序的性能;设计的目的是为了处理那些可以被递归拆分的任务。
This concise book empowers all Java developers to master the complexity of the Java thread APIs and concurrency utilities. This knowledge aids the Java developer in writing correct and complex ...