dependency scanner

  • Follow


I am looking for a class/program that will scan a .java file for imports 
  (recursively) to discover which ones need to be rebuilt.  This is 
close to GNU's makedepend or cook's c_incl.
0
Reply Aryeh.Friedman (91) 5/11/2008 3:03:34 PM

"Aryeh M. Friedman" <aryeh.friedman@gmail.com> writes:
>I am looking for a class/program that will scan a .java file for imports 
>  (recursively) to discover which ones need to be rebuilt.  This is 
>close to GNU's makedepend or cook's c_incl.

  You might try to trace the classes loaded by the JVM.

http://www.md.pp.ru/~eu/jdk6options.html#TraceClassLoadingPreorder
http://www.md.pp.ru/~eu/jdk6options.html#TraceClassLoading

0
Reply ram (2829) 5/11/2008 3:14:17 PM


Aryeh M. Friedman wrote:
> I am looking for a class/program that will scan a .java file for imports 
>  (recursively) to discover which ones need to be rebuilt.  This is close 
> to GNU's makedepend or cook's c_incl.

javac

-- 
Lew
0
Reply lew (2143) 5/11/2008 3:24:08 PM

Aryeh M. Friedman wrote:
> I am looking for a class/program that will scan a .java file for imports 
>  (recursively) to discover which ones need to be rebuilt.  This is close 
> to GNU's makedepend or cook's c_incl.

You don't use that type of tools with Java.

The javac compiler itself compiles missing parts.

And the ant tool (which you should use for build) checks what to
rebuild based on dates similar to ant.

Just use ant and javac and forget about dependencies.

If you use maven for build it will even be able to download
external packages you need for you.

Arne
0
Reply arne6 (9487) 5/11/2008 5:19:25 PM

Arne Vajh�j wrote:
> Aryeh M. Friedman wrote:
>> I am looking for a class/program that will scan a .java file for
>>  imports (recursively) to discover which ones need to be rebuilt. 
>> This is close to GNU's makedepend or cook's c_incl.
>
> You don't use that type of tools with Java.
>
> The javac compiler itself compiles missing parts.
>
> And the ant tool (which you should use for build) checks what to
> rebuild based on dates similar to ant.
>
> Just use ant and javac and forget about dependencies.

And when that's not good enough (like when superclass change, require 
subclasses to recompile too), curse a bit, delete all the .class 
files, and rebuild the whole thing.


0
Reply mscottschilling (1976) 5/11/2008 6:57:54 PM

Mike Schilling wrote:
> Arne Vajh�j wrote:
>> Aryeh M. Friedman wrote:
>>> I am looking for a class/program that will scan a .java file for
>>>  imports (recursively) to discover which ones need to be rebuilt. 
>>> This is close to GNU's makedepend or cook's c_incl.
>> You don't use that type of tools with Java.
>>
>> The javac compiler itself compiles missing parts.
>>
>> And the ant tool (which you should use for build) checks what to
>> rebuild based on dates similar to ant.
>>
>> Just use ant and javac and forget about dependencies.
> 
> And when that's not good enough (like when superclass change, require 
> subclasses to recompile too), curse a bit, delete all the .class 
> files, and rebuild the whole thing.

Many people make a clean target in build.xml !

Arne
0
Reply arne6 (9487) 5/11/2008 6:59:07 PM

> The javac compiler itself compiles missing parts.
> 
> And the ant tool (which you should use for build) checks what to
> rebuild based on dates similar to ant.
> 
> Just use ant and javac and forget about dependencies.

Compiling based on dates isn't enough. The most anoying problem (but not 
the only one) during compilation is inlining constants. The only way to 
be 100% sure of proper compilation is to allways recompile all sources. 
Fortunatelly recompilation based on dates works often enought to make 
ant/maven quite usefull tools.
0
Reply BardzoTajneKonto (54) 5/11/2008 7:08:10 PM

Arne Vajh�j wrote:
> Mike Schilling wrote:
>> Arne Vajh�j wrote:
>>> Aryeh M. Friedman wrote:
>>>> I am looking for a class/program that will scan a .java file for
>>>>  imports (recursively) to discover which ones need to be rebuilt.
>>>> This is close to GNU's makedepend or cook's c_incl.
>>> You don't use that type of tools with Java.
>>>
>>> The javac compiler itself compiles missing parts.
>>>
>>> And the ant tool (which you should use for build) checks what to
>>> rebuild based on dates similar to ant.
>>>
>>> Just use ant and javac and forget about dependencies.
>>
>> And when that's not good enough (like when superclass change, 
>> require
>> subclasses to recompile too), curse a bit, delete all the .class
>> files, and rebuild the whole thing.
>
> Many people make a clean target in build.xml !

I always do, for several reasons; the above is one of them.  My 
comment was a complaint about the fact that the <javac> task is 
*almost* good enough to handle dependencies, but not quite.  When I, 
for instance, get a batch of newer source code from the SCM system, I 
always do a clean build, since there's no way to be sure it isn't 
necessary, and I don't want to waste time with spurious problems from 
not building clean when it was required. 


0
Reply mscottschilling (1976) 5/11/2008 7:09:55 PM

> I am looking for a class/program that will scan a .java file for imports 
>  (recursively) to discover which ones need to be rebuilt.  This is close 
> to GNU's makedepend or cook's c_incl.

You didn't consider import x.y.* and classes in the same package. You'll 
have to parse the whole source to do this properly (the most popular 
tool is javacc) - it may not be much faster that simply recompiling 
those sources.
0
Reply BardzoTajneKonto (54) 5/11/2008 7:14:29 PM

jolz wrote:
>> I am looking for a class/program that will scan a .java file for
>>  imports (recursively) to discover which ones need to be rebuilt. 
>> This is close to GNU's makedepend or cook's c_incl.
>
> You didn't consider import x.y.* and classes in the same package.

And the use of fully qualified named with no import. 


0
Reply mscottschilling (1976) 5/11/2008 7:31:48 PM

Mike Schilling wrote:
>>> ... recompile too), curse a bit, delete all the .class
>>> files, and rebuild the whole thing.

Arne Vajh�j wrote:
>> Many people make a clean target in build.xml !

Mike Schilling wrote:
> I always do, for several reasons; the above is one of them.  My 
> comment was a complaint about the fact that the <javac> task is 
> *almost* good enough to handle dependencies, but not quite.  When I, 
> for instance, get a batch of newer source code from the SCM system, I 
> always do a clean build, since there's no way to be sure it isn't 
> necessary, and I don't want to waste time with spurious problems from 
> not building clean when it was required. 

Lending weight to that already well-founded conclusion, if you use an IDE it 
often really helps clear up the IDE's own internal state and coherence after a 
fresh checkout (or update) to do a clean build.

-- 
Lew
0
Reply lew (2143) 5/11/2008 7:49:11 PM

Mike Schilling wrote:
> jolz wrote:
>>> I am looking for a class/program that will scan a .java file for
>>>  imports (recursively) to discover which ones need to be rebuilt. 
>>> This is close to GNU's makedepend or cook's c_incl.
>> You didn't consider import x.y.* and classes in the same package.
> 
> And the use of fully qualified named with no import. 

Given that 'import' is just syntactic sugar for FQNs, any dependency analysis 
tool would only use FQNs for its analysis, I should think.

-- 
Lew
0
Reply lew (2143) 5/11/2008 7:51:55 PM

Mike Schilling wrote:
> Arne Vajh�j wrote:
>> Mike Schilling wrote:
>>> Arne Vajh�j wrote:
>>>> Aryeh M. Friedman wrote:
>>>>> I am looking for a class/program that will scan a .java file for
>>>>>  imports (recursively) to discover which ones need to be rebuilt.
>>>>> This is close to GNU's makedepend or cook's c_incl.
>>>> You don't use that type of tools with Java.
>>>>
>>>> The javac compiler itself compiles missing parts.
>>>>
>>>> And the ant tool (which you should use for build) checks what to
>>>> rebuild based on dates similar to ant.
>>>>
>>>> Just use ant and javac and forget about dependencies.
>>> And when that's not good enough (like when superclass change, 
>>> require
>>> subclasses to recompile too), curse a bit, delete all the .class
>>> files, and rebuild the whole thing.
>> Many people make a clean target in build.xml !
> 
> I always do, for several reasons; the above is one of them.  My 
> comment was a complaint about the fact that the <javac> task is 
> *almost* good enough to handle dependencies, but not quite.  When I, 
> for instance, get a batch of newer source code from the SCM system, I 
> always do a clean build, since there's no way to be sure it isn't 
> necessary, and I don't want to waste time with spurious problems from 
> not building clean when it was required. 

And usually Java builds are reasonable fast even when building from
scratch.

Arne
0
Reply arne6 (9487) 5/11/2008 9:35:14 PM

Arne Vajhøj wrote:
> And usually Java builds are reasonable fast even when building from
> scratch.

If they aren't, it's an indicator that it's time to refactor portions of the 
project into independent JARs.

-- 
Lew
0
Reply lew (2143) 5/11/2008 10:38:56 PM

Lew wrote:
> Mike Schilling wrote:
>> jolz wrote:
>>>> I am looking for a class/program that will scan a .java file for
>>>>  imports (recursively) to discover which ones need to be rebuilt.
>>>> This is close to GNU's makedepend or cook's c_incl.
>>> You didn't consider import x.y.* and classes in the same package.
>>
>> And the use of fully qualified named with no import.
>
> Given that 'import' is just syntactic sugar for FQNs, any dependency
> analysis tool would only use FQNs for its analysis, I should think.

Certainly.  But the OP was suggesting a tool that scanned the .java 
file lexically for imports, and that would fail to find

    p1.p2.p3.class1 c = null;

Personally, I'd build a dependency tool that scanned the .class file, 
not the .java file.  (If there is no .class file, you don't need to 
know anything about dependencies to realize that the .java file needs 
to be recompiled.)  The bytecode has everything you need, and there is 
lots of open-source code out there to help you parse it. 


0
Reply mscottschilling (1976) 5/12/2008 4:41:37 AM

Mike Schilling wrote:
> Lew wrote:
>> Mike Schilling wrote:
>>> jolz wrote:
>>>>> I am looking for a class/program that will scan a .java file for
>>>>>  imports (recursively) to discover which ones need to be rebuilt.
>>>>> This is close to GNU's makedepend or cook's c_incl.
>>>> You didn't consider import x.y.* and classes in the same package.
>>> And the use of fully qualified named with no import.
>> Given that 'import' is just syntactic sugar for FQNs, any dependency
>> analysis tool would only use FQNs for its analysis, I should think.
> 
> Certainly.  But the OP was suggesting a tool that scanned the .java 
> file lexically for imports, and that would fail to find
> 
>     p1.p2.p3.class1 c = null;
> 
> Personally, I'd build a dependency tool that scanned the .class file, 
> not the .java file.  (If there is no .class file, you don't need to 
> know anything about dependencies to realize that the .java file needs 
> to be recompiled.)  The bytecode has everything you need, and there is 
> lots of open-source code out there to help you parse it. 

Doesn't every existing tool that does dependency scanning work off the .class 
files?

It escaped me that the question restricted the scan to source files.  I simply 
figured any dependency tool would simply have to work off the .class files, 
never even dreaming that anyone would try to do it from the source.

-- 
Lew
0
Reply lew (2143) 5/12/2008 11:52:27 AM

Lew wrote:
> Mike Schilling wrote:
>> Lew wrote:
>>> Mike Schilling wrote:
>>>> jolz wrote:
>>>>>> I am looking for a class/program that will scan a .java file for
>>>>>>  imports (recursively) to discover which ones need to be rebuilt.
>>>>>> This is close to GNU's makedepend or cook's c_incl.
>>>>> You didn't consider import x.y.* and classes in the same package.
>>>> And the use of fully qualified named with no import.
>>> Given that 'import' is just syntactic sugar for FQNs, any dependency
>>> analysis tool would only use FQNs for its analysis, I should think.
>>
>> Certainly.  But the OP was suggesting a tool that scanned the .java 
>> file lexically for imports, and that would fail to find
>>
>>     p1.p2.p3.class1 c = null;
>>
>> Personally, I'd build a dependency tool that scanned the .class file, 
>> not the .java file.  (If there is no .class file, you don't need to 
>> know anything about dependencies to realize that the .java file needs 
>> to be recompiled.)  The bytecode has everything you need, and there is 
>> lots of open-source code out there to help you parse it. 
> 
> Doesn't every existing tool that does dependency scanning work off the 
> .class files?
> 
> It escaped me that the question restricted the scan to source files.  I 
> simply figured any dependency tool would simply have to work off the 
> .class files, never even dreaming that anyone would try to do it from 
> the source.
> 

Isn't this a catch-22 though in that if you are in a mixed lang 
enviroment and either need the java compiled first (in the case of it 
being the backend) or the other lang  being done first (JNI) and it is 
being compiled by someone who has not developed it then it then follows 
there are no .class files.   Why did sun get rid of -depend on javac 
anyways?  I also found JavaDepend 
(http://ptolemy.eecs.berkeley.edu/~cxh/java/javadepend/) but it doesn't 
work on anything newer then 1.1 and I am doing 1.5 projects.

Also in an other post in the thread someone mentioned if a compile 
doesn't work with say javac `find . -name '*.java'` or javac *.java then 
it is time to break it apart into standalone jar's.   This too raises a 
depend issue in that if oyu have several packages which ones need to be 
recompiled when X changes.

In short I am *NOT IMPRESSED* with Java's ability to manage large 
projects,   In a way this is the same issue recursive make has 
http://aegis.sourceforge.net/auug97.pdf
0
Reply Aryeh.Friedman (91) 5/12/2008 12:32:25 PM

Aryeh M. Friedman wrote:
> In short I am *NOT IMPRESSED* with Java's ability to manage large 
> projects,   In a way this is the same issue recursive make has 
> http://aegis.sourceforge.net/auug97.pdf

Funny, I don't remember anyone claiming that Java is a project-management 
system.  I thought it was a programming language.  Do you blame C for not 
having the features of make, I wonder?

Use Ant, of course.  Maven for things Ant can't.

-- 
Lew
0
Reply lew (2143) 5/12/2008 12:40:49 PM

On Sun, 11 May 2008 11:03:34 -0400, "Aryeh M. Friedman"
<aryeh.friedman@gmail.com> wrote, quoted or indirectly quoted someone
who said :

>I am looking for a class/program that will scan a .java file for imports 

Imports are not sufficient since you can have fully qualified names in
the source text.

Further imports such as import javax.swing.* brings in much more than
you would actually need.

If you scanned the class files, you would miss new classes referenced
since the last compile.

So that means you have to scan source and parse it.  JavaC can
probably do that faster than you can.

I think you are better off to:

1. use ANT so Javac does not need to reloaded for multiple compiles.

2. just feel the whole shebang on the command line and let JavaC
figure out what needs to be recompiled.

3. from time to time do a clean compile (erase all class files), and
certainly before any release.
-- 

Roedy Green Canadian Mind Products
The Java Glossary
http://mindprod.com
0
Reply see_website (4858) 5/12/2008 12:49:46 PM

Lew wrote:
> Aryeh M. Friedman wrote:
>> In short I am *NOT IMPRESSED* with Java's ability to manage large 
>> projects,   In a way this is the same issue recursive make has 
>> http://aegis.sourceforge.net/auug97.pdf
> 
> Funny, I don't remember anyone claiming that Java is a 
> project-management system.  I thought it was a programming language.  Do 
> you blame C for not having the features of make, I wonder?

C does at least has a trivial way to find depends (#include)

> 
> Use Ant, of course.  Maven for things Ant can't.
> 

Neither seems to be able to manage non-Java projects and if one wants to 
have the project a part of a OS dist (or ports collection) then almost 
all of them only accept make compatible build scripts.   An other 
requirement I have is the ability to stich the source tree together from 
disjoint dirs (i.e. some of it is in a repo (the unmodified code) and 
some is in a local dir (the modified code)).   The only dependency 
management tool I have ever found that manages this in anything like a 
reasonable manner is cook 
(http://miller.emu.id.au/pmiller/software/cook/) probally because it was 
written by the same person that wrote my SCM tool (aegis.sf.net).   The 
only other alternative I know of is something like unionfs which is 
horrible.  Oh forgot to mention cook also has an builtin Makefile 
generator in it so the OS ports/dist is happy.
0
Reply Aryeh.Friedman (91) 5/12/2008 12:53:19 PM

Aryeh M. Friedman wrote:
> Lew wrote:
>> Aryeh M. Friedman wrote:
>>> In short I am *NOT IMPRESSED* with Java's ability to manage large 
>>> projects,   In a way this is the same issue recursive make has 
>>> http://aegis.sourceforge.net/auug97.pdf
>>
>> Funny, I don't remember anyone claiming that Java is a 
>> project-management system.  I thought it was a programming language.  
>> Do you blame C for not having the features of make, I wonder?
> 
> C does at least has a trivial way to find depends (#include)

Huh?  What does #include have to do with managing object file dependencies?

If you recompile foo.c and bar.o depends on foo.o, nothing in C is going to 
help you figure that one out, via #include or otherwise.  You need make for that.

#include is a *pre*-processing step; it happens before the compilation even 
looks at the source code.  There is no equivalent in Java.  The 'import' 
directive does not include anything at all; it just helps the compiler with 
name resolution.

-- 
Lew
0
Reply lew (2143) 5/12/2008 2:21:57 PM

Lew wrote:
> Aryeh M. Friedman wrote:
>> C does at least has a trivial way to find depends (#include)
> 
> Huh?  What does #include have to do with managing object file dependencies?
> 
> If you recompile foo.c and bar.o depends on foo.o, nothing in C is going 
> to help you figure that one out, via #include or otherwise.  You need 
> make for that.
> 
> #include is a *pre*-processing step; it happens before the compilation 
> even looks at the source code.  There is no equivalent in Java.  The 
> 'import' directive does not include anything at all; it just helps the 
> compiler with name resolution.

What he means is that some compilers (such as gcc) have support for 
generating dependency information based on #include statements. IIRC the 
-M flag will cause gcc to write out a file describing what objects 
depend on what source files based on the knowledge it obtains through 
parsing. BTW there used to be a similar tool for Java (JavaDeps?) but I 
have no idea if it's still around.

RM
0
Reply rexm (52) 5/12/2008 2:43:30 PM

Rex Mottram wrote:
> Lew wrote:
>> Aryeh M. Friedman wrote:
>>> C does at least has a trivial way to find depends (#include)
>>
>> Huh?  What does #include have to do with managing object file 
>> dependencies?
>>
>> If you recompile foo.c and bar.o depends on foo.o, nothing in C is 
>> going to help you figure that one out, via #include or otherwise.  You 
>> need make for that.
>>
>> #include is a *pre*-processing step; it happens before the compilation 
>> even looks at the source code.  There is no equivalent in Java.  The 
>> 'import' directive does not include anything at all; it just helps the 
>> compiler with name resolution.
> 
> What he means is that some compilers (such as gcc) have support for 
> generating dependency information based on #include statements. IIRC the 
> -M flag will cause gcc to write out a file describing what objects 
> depend on what source files based on the knowledge it obtains through 
> parsing. BTW there used to be a similar tool for Java (JavaDeps?) but I 
> have no idea if it's still around.

In Java you would accomplish that most easily by compiling the source 
(equivalent to "knowledge it obtains through parsing" in your example), using 
the built-in compilation libraries if being done programmatically, then 
building your dependency tree off the declarations in the bytecode, which will 
all use FQNs.

-- 
Lew
0
Reply lew (2143) 5/12/2008 2:50:03 PM

Lew wrote:
> Aryeh M. Friedman wrote:
>> Lew wrote:
>>> Aryeh M. Friedman wrote:
>>>> In short I am *NOT IMPRESSED* with Java's ability to manage large 
>>>> projects,   In a way this is the same issue recursive make has 
>>>> http://aegis.sourceforge.net/auug97.pdf
>>>
>>> Funny, I don't remember anyone claiming that Java is a 
>>> project-management system.  I thought it was a programming language.  
>>> Do you blame C for not having the features of make, I wonder?
>>
>> C does at least has a trivial way to find depends (#include)
> 
> Huh?  What does #include have to do with managing object file dependencies?


As with any type of build procedure there are implicit and explicit 
depends that make/cook/whatever needs to handle...  Explicit ones are 
the ones you are thinking of when you are thinking that foo.c depends on 
ack.c because it uses bar() from ack.c... *BUT* since ANSI (and almost 
all compilors accept ANSI by default now) requires a prototype (even if 
it returns an int or void) this means I need to define the prototype for 
bar() somewhere and more often then not you do that in ack.h... now in 
foo.c I have:

	#include "ack.h"

Assuming that I use the same name for the .h and .c it goes to I only 
need to scan for #include's in foo.c and do a batch processed search and 
replace (say with sed in unix) and replace all .h with .c (or actually 
..o for techinical reasons)... this now gives me a list of all my 
implicit depends (due to the need to prototype about 95% of the total 
for a typical project)... All I need to do in the 
makefile/cookbook/whatever is add a rule like this (cook has nicer 
syntax then make so I will use it):

	%0%.o: %0%.c			
	/*in reality I would also add a call to the dep scanner above*/
	{
		gcc [command line opts] -c %0%.c -o [target];
	}

You should be asking now how to handle explicit depends... thats when 
you actually hard/softcode it into the makefile/cookbook/whatever with 
something like:

	bin/foo: lib/liback.a
	{
		gcc -Llib foo.c -o [target];
	}

	lib/liback.a: lib/ack.o lib/fred.o ....
	{
              ar rv [target] ack.o fred.o ...
	}

Almost all these can be soft coded by various tricks with a good depend 
scanner for example to get the "ack.o fred.o ..." list I would just say 
every .c in lib/ needs to become a .o.


Now cook does something really smart at this point compared to make.   I 
  creates the entire DAG *BEFORE* attempting to build any target thus 
allows you to avoid all the issues in RMCF.

Due to this premade DAG approach I can use the same cookbook to build 
and package multiple packages (using the java term) *BUT* only when 
something in them or a package they depend on changes.

The reason for the orginal question is I do not want to have to use 
nothing but explicit depends and if I am writting this for public 
release the person who DL's the source code will not have any .class 
files yet.

> 
> If you recompile foo.c and bar.o depends on foo.o, nothing in C is going 
> to help you figure that one out, via #include or otherwise.  You need 
> make for that.
> 
> #include is a *pre*-processing step; it happens before the compilation 
> even looks at the source code.  There is no equivalent in Java.  The 
> 'import' directive does not include anything at all; it just helps the 
> compiler with name resolution.
> 
0
Reply Aryeh.Friedman (91) 5/12/2008 2:56:26 PM

Aryeh M. Friedman wrote:

>> Use Ant, of course.  Maven for things Ant can't.
>>
>
> Neither seems to be able to manage non-Java projects and if one wants
> to have the project a part of a OS dist (or ports collection) then
> almost all of them only accept make compatible build scripts.

Ant can run any java program or native exceutable you like.  We use it to 
build a combined Java and C# system.  In fact, the standard Ant distribution 
includes tasks to build .NET programs, though we've had to modify them for 
our requirements.


0
Reply mscottschilling (1976) 5/12/2008 4:06:50 PM

Lew wrote:
>> Certainly.  But the OP was suggesting a tool that scanned the .java
>> file lexically for imports, and that would fail to find
>>
>>     p1.p2.p3.class1 c = null;
>>
>> Personally, I'd build a dependency tool that scanned the .class file,
>> not the .java file.  (If there is no .class file, you don't need to
>> know anything about dependencies to realize that the .java file needs
>> to be recompiled.)  The bytecode has everything you need, and there
>> is lots of open-source code out there to help you parse it.
>
> Doesn't every existing tool that does dependency scanning work off
> the .class files?
>
> It escaped me that the question restricted the scan to source files. I 
> simply figured any dependency tool would simply have to work off
> the .class files, never even dreaming that anyone would try to do it
> from the source.

There are more things in Heaven and Earth, Horatio, than are dreamt of in 
your philosphy. :-) 


0
Reply mscottschilling (1976) 5/12/2008 4:10:30 PM

Aryeh M. Friedman wrote:
> In short I am *NOT IMPRESSED* with Java's ability to manage large 
> projects,

How many years of experience do you have using ant ?

Arne
0
Reply arne6 (9487) 5/12/2008 10:28:49 PM

Mike Schilling wrote:
> There are more things in Heaven and Earth, Horatio, than are dreamt of in 
> your philosphy. :-) 

That's why I keep coming to the newsgroup, Poor Yorick.  I learn new stuff all 
the time.

-- 
Lew
0
Reply lew (2143) 5/13/2008 2:19:03 AM

I manage the page with javadepend, which has not been updated in
years.  I stashed a copy of javadepend there mainly to preserve
history.

We had a similar problem where we needed to find dependencies and
someone use Ivy, which seemed to work reasonably well.

http://java-source.net/open-source/build-systems/ivy

Yes, it is always possible to miss classes because of reflection, but
Ivy gave us a leg up on analyzing dependencies.  If you really want to
know what packages are used, do treeshaking by running java -verbose
and exercise the program.  We use this to quite effectively for
shipping small jars.  However, it is difficult to do treeshaking
automatically and get a complete set of jars unless you effectively
exercise the program.

Also, the Eclipse Metrics package has a dependency visualizer that
might be of interest.  I looked at it briefly this week.

About the ant/make thing.  After using both and shipping both, I've
come to the conclusion that that Ant and Make to be pretty much
functionally equivalent.  One can argue about the details though.

To ship software, what I need is the following:
1. Before compilation, some checks and searches need to happen.
   Checks for the right version of the compiler etc., searches for
   extensions.  Historically, this is done with configure, though ant
   can do more of this than it used to.
   Unfortunately, portions of my user community does not want to run
   configure before starting Eclipse.  Nor do they want to run an
   ant task from within Eclipse.
2. During compilation, packages that won't compile need to be skipped.
   Many years ago, ant could not handle dependencies, now it can.
   Also, many years ago, the ant page said it was written by people
   who did not understand make.  I've seen this issue before when
   people complain about make and then start writing their own system.        
3. At release time, each package needs to have some sort of manifest
   that lists files and subdirectories to be shipped.  Currently, the
   manifest for Ptolemy is in makefiles, but it could be in separate text
   files, which could be read by recursive ant files.
4. At user run time, I'd like to provide modules and provided updates 
   similar to Eclipse/Equinox and the oSGI work.  This requires dependency
   analysis based on versions, including supporting modules that
   themselves might require different versions of _the_same_ jar file.
   This is difficult, and requires classpath support.
         

The manifest files are what refute some of the arguments put forth in
"recursive make considered harmful."  

Ant and Eclipse's "Just compile everything" strategy does not work for
research projects where some packages in a tree will not compile
unless missing dependencies are present.

Thus, I'm interested in Maven, but really looking for something that
handles complex configuration and package dependencies.

Personally, I'm pretty happy with the package system, it does make
modularizing large projects (>300k LOC) possible.  However, more
work needs to be done concerning supporting multiple projects.

_Christopher


"Aryeh M. Friedman" <aryeh.friedman@gmail.com> writes:

> Lew wrote:
> > Mike Schilling wrote:
> >> Lew wrote:
> >>> Mike Schilling wrote:
> >>>> jolz wrote:
> >>>>>> I am looking for a class/program that will scan a .java file for
> >>>>>>  imports (recursively) to discover which ones need to be rebuilt.
> >>>>>> This is close to GNU's makedepend or cook's c_incl.
> >>>>> You didn't consider import x.y.* and classes in the same package.
> >>>> And the use of fully qualified named with no import.
> >>> Given that 'import' is just syntactic sugar for FQNs, any dependency
> >>> analysis tool would only use FQNs for its analysis, I should think.
> >>
> >> Certainly.  But the OP was suggesting a tool that scanned the .java
> >> file lexically for imports, and that would fail to find
> >>
> >>     p1.p2.p3.class1 c = null;
> >>
> >> Personally, I'd build a dependency tool that scanned the .class
> >> file, not the .java file.  (If there is no .class file, you don't
> >> need to know anything about dependencies to realize that the .java
> >> file needs to be recompiled.)  The bytecode has everything you
> >> need, and there is lots of open-source code out there to help you
> >> parse it.
> > Doesn't every existing tool that does dependency scanning work off
> > the .class files?
> > It escaped me that the question restricted the scan to source files.
> > I simply figured any dependency tool would simply have to work off
> > the .class files, never even dreaming that anyone would try to do it
> > from the source.
> >
> 
> Isn't this a catch-22 though in that if you are in a mixed lang
> enviroment and either need the java compiled first (in the case of it
> being the backend) or the other lang  being done first (JNI) and it is
> being compiled by someone who has not developed it then it then
> follows there are no .class files.   Why did sun get rid of -depend on
> javac anyways?  I also found JavaDepend
> (http://ptolemy.eecs.berkeley.edu/~cxh/java/javadepend/) but it
> doesn't work on anything newer then 1.1 and I am doing 1.5 projects.
> 
> Also in an other post in the thread someone mentioned if a compile
> doesn't work with say javac `find . -name '*.java'` or javac *.java
> then it is time to break it apart into standalone jar's.   This too
> raises a depend issue in that if oyu have several packages which ones
> need to be recompiled when X changes.
> 
> In short I am *NOT IMPRESSED* with Java's ability to manage large
> projects,   In a way this is the same issue recursive make has
> http://aegis.sourceforge.net/auug97.pdf

-- 
Christopher Brooks (cxh at eecs berkeley edu) University of California
Chess Executive Director                      US Mail: 337 Cory Hall #1774
Programmer/Analyst Chess/Ptolemy/Trust        Berkeley, CA 94720-1774
ph: 510.643.9841 fax:510.642.2718	      (office: 400A Cory)
home: (F-Tu) 707.665.0131 (W-F) 510.655.5480  
0
Reply cxh (8) 5/16/2008 3:07:51 PM

28 Replies
26 Views

(page loaded in 0.317 seconds)


Reply: