How are people reigning in 3rd party libraries? Selecting which pakcages are suitable for use, which licenses? How does your Software Process identify when someone uses un-approved software/licenses?
1. Martin Hache (Senior Technical Java J2EE Consultant at HP)
Terrific question, I'm surprised no one's responded in the 24 hours since it's been posted. I'll tell you what we did but I would not exactly call it a solution, also, I found the problem to be larger than what you described in your post.
Before I relate my experience, I would suggest that you look at Maven2, I hear that it has a system to manage library dependencies; I've never used it but those who love it, really love it.
The problem for us (a web development team of 15 or so individuals working on a few dozen web apps) was not only 3rd party libraries but also the 3rd party libraries that 3rd party libraries used (4th party?). E.g. When the Spring JAR uses some Apache Common's JARS. The versions of these libraries would clash with the versions our applications wanted to use. This was particularly apparent in our own reusable components which could be shared across several applications. These amounted to 3rd party libraries with 3rd party libraries of their own.
Unfortunately, we never did crack this nut to my satisfaction, we settled for establishing a few guidelines. We added version numbers to JARs if they didn't have them (so spring.jar became spring-2.0.2.jar) this allowed us to ID the version of a library with a simple look. On top of that we basically leveraged our build order: components distributed the jars they needed to compile to the child apps/components that depended on them. Those dependents modules didn't usually contain the JARs if one of the components they depended on distributed it. If a child app/component needed a newer JAR than what the parent component was offering then we would create a new version of the parent with the new version of the JAR. Clashes were handled manually, by talking through them.
Your question may have more to do with licensing and authority to use a library, but we didn't do much of that, again a manual vetting of the Jar and licensing is what we did.
2. Curt Yanko （Sr IS Architect at UHG）
I have left it *open-ended* since I'm fishing a bit here and don't want to influence the answers too much.
I am indeed talking about the full monty as-it-were, and am interested in creating a Definitive Software LIbrary of *approved* components and then contraining the build system to just those. Additionaly, failing a build should I see a license that scares me, I'm looking at you Affero!
Maven is indeed central to our strategy. Site reports and their BOM's play a key role in at least getting visibility. Now I want control.
3. Ben Weatherall （Configuration Manager at PDX, Inc.）
First, I want to point out a commercial solution from OpenLogic called OLEX. It does the license analysis (are these licenses compatible?) and license obligations (if you use this, in this fashion, you must do ...). It also has functionality to scan both source and binaries to determine which FOSS components are included, whether you intended for them to be or not. And no, I am not associated with OpenLogic - just paranoid enough to be checking them out.
Now, as to what we actually (try to) do, whenever someone decides to add a third party component to the mix, they are required to submit links to the "owner" and to wherever they acquired the binaries. They are then allowed to commit to the primary repository only those binaries that are actually built from the component source. All of the other dependent components that may be supplied "for convenience" must be checked that they are available from an "owner" and then the process recurses.
I enforce this, otherwise no license checks would ever be done and there would be no way I could "escrow" the source in case of legal challenge. Periodically, the architecture team, product managers and CM get together to review changes to our third party repository and how they are being used. We try to distribute the work since it is very time intensive to track everything back to the "owner" level and verify it.
Curt, we do not use Maven for the very reason you like it - it will find what it needs even if I don't know about it. I am not totally happy with our solution since it is so manually intensive, but it is what I have today. It will evolve. Either that or I need to revoke Development's right to update third party components and allocate that function to an already overloaded team.
We get our BOMs, etc. from a use of AccuRev, cvs and AnthillPro. The combination has kept me out of too much trouble so far.
--
Elian
Configuration Manage Engineer
MSN: smallfish961@hotmail.com
Email: smallfish382+work@gmail.com
Tuesday, July 12, 2011
Thursday, January 13, 2011
Hudson报错：java.lang.OutOfMemoryError: Java heap space
Hudson报错：java.lang.OutOfMemoryError: Java heap space
问题: 触发hudson构建，个别仓库会出现如下的错误信息:
hudson.util.IOException2: remote file operation failed: E:\WORK_DATA\Build\Build_Src\project\project1 at hudson.remoting.Channel@1a0c382:239.161
at hudson.FilePath.act(FilePath.java:753)
at hudson.FilePath.act(FilePath.java:735)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:653)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:601)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1038)
at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:481)
at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:413)
at hudson.model.Run.run(Run.java:1259)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:47)
at hudson.model.ResourceController.execute(ResourceController.java:95)
at hudson.model.Executor.run(Executor.java:129)
Caused by: java.io.IOException: Remote call on 239.161 failed
at hudson.remoting.Channel.call(Channel.java:573)
at hudson.FilePath.act(FilePath.java:744)
... 10 more
Caused by: java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOfRange(Unknown Source)
at java.lang.String.<init>(Unknown Source)
分析测试:
1. "java.lang.OutOfMemoryError: Java heap space"
有很多资料显示此错误信息跟Tomcat内存大小的设置有关,但实际作了调整后问题并没得到解决。也尝试增加了物理内存，由原来2G提升为4G，并更换操作系统为64位的。即64位win2008+jdk1.6.0_22+tomcat-6.0.29+hudson1.386，以下为测试现象：
1) 第一天借了两条2G的内存,装上了64位/win2008的系统;对系统update后,装了些软件,搭建了tomcat服务.
2) 第二天准备远程连接操作,发现机子挂了.到机房一看,机子自动重启,并报错:注册表失败,需要修复的信息.
3) 打算通过光盘启动来做系统修复,结果通过光盘加载完就出现蓝屏.---这现象,开始怀疑跟那两条2G的内存有关系..硬件管理员帮忙试验了一下内存。后面发现其实是有两个系统存在.奈闷,昨天重装系统时,只读到一个1T的硬盘.这问题先不纠结了,把有效的系统盘设为优先启动,正常进入.
4) 系统正常启动后,远程操作时,结果每隔几十分钟就会自动重启.根本无法正常用.
5) 继续怀疑内存不兼容的.到机房,把旧的内存撤掉,留下那两条2G的,发现蓝屏启动不起来.只留任一条,也是如此现象.后面,直接更换了内存,换了两条新,又再次正常启动.
6) 系统正常启动后,远程操作时,突然发现机子还是每几十分钟后就会自动重启. 在中午12点到2点的时间大约重启了三四回.2点后,又进机房关察了一下,确没发现有什么异常现象.
7) 继续操作,通过startup.bat启动的Tomcat服务,跑了一下任务,结果发现过一会系统自动把startup.bat给关了.在持续一会,机子又出现了自动重启的现象.等进去都会弹出一个对话框:windows已从异常关机中恢复.
8) 撤掉两条旧的1G宽内存,保留一条宽一条窄的2G内存,自动重启现象仍存在
9) 保留两条旧的1G宽内存,撤掉一条宽一条窄的2G内存,系统运行正常
10) 保留两条旧的1G宽内存+一条宽的2内存,系统运行正常
2. hudson.scm.SubversionSCM.checkout
1) 试着把job放在Master上跑,结果一切正常。如果换到指定Slave机上，每次执行到取源代码就报错了.难道问题出在Master与Slave的交互？？
2) 继续试着把Source Code Management的配置去掉，直接调用slave上的nant脚本取代码动作,一切运行正常的？？这不禁让我认为跟hudson自带的SVN功能有问题？但比较疑惑的是有些SVN仓库有这样的现象，有些仓库又不会。出问题的仓库有个显著区别是添加比较多的svn:externals属性，难道这跟externals设了太多也有关系？
结论:
经过一番测试后，最简便的调整Tomcat内存大小并不是解决问题的办法,但不管怎么说这个处理是应该优先考虑的。目前这个问题的本质原因可能跟hudson自带的SVN有关。现能解决的是对存在问题的job不使用Source Code Management配置,直接用后台脚本替代。
后序:
1. 关于JAVA_OPTS的设置
在调整TOMCAT内存大小中,几回设置JAVA_OPTS值才发现，-Xmx与MaxPermSize两个值是相铺相成的,以下几组是物理内存为2G的配置,请参考：
set JAVA_OPTS=-Xms1500m -Xmx1500m -XX:PermSize=128M -XX:MaxPermSize=256M ,此配置无效
set JAVA_OPTS=-Xms1300m -Xmx1300m -XX:PermSize=128M -XX:MaxPermSize=256M ,此配置OK
set JAVA_OPTS=-Xms1500m -Xmx1500m -XX:PermSize=64M -XX:MaxPermSize=128M OK,此配置OK
set JAVA_OPTS=-Xms1500m -Xmx1500m -XX:PermSize=64M -XX:MaxPermSize=150M OK,此配置OK
2. 问题反馈官方地址
--
Elian
Configuration Manage Engineer
MSN: smallfish961@hotmail.com
Email: smallfish382+work@gmail.com
Subscribe to:
Posts (Atom)
Posts
