Boost.Build Projects

BOOST WIKI | RecentChanges | Preferences | Page List | Links List

Projects and targets

Note: this is not yet complete proposal. Comments are very welcome.

Goals

We'd like to have the following functionality:

It should be possible to freely use artifacts from other projects, regardless of whether those project use Boost.Build for their construction.
Artifacts used should be able to propagate some information back to targets that use them, for example:
- Using a dynamic library should affect library search paths,
- Using a library might affect include paths.
When artefacts are part of Boost.Build managed project, dependencies should be correctly tracked, for example, one might use a tool which itself is under development. Then, changes made to the tool's source must result in reconstruction of all targets made with the tool.

Project declarations

Each project is some directory tree, containing Jamfiles. That tree becames an integral entity by means of three rules: project, subproject, and subinclude.

The rule project is used to declare a project's name:

 project boost ;

It can be used only in top-level Jamfile. Note: Will we need an ability to find the name of the current project? Very likely. In this case, we'd have to require that project rule appear in Jamrules.

Sure, and it would not be a bad idea to consider renaming Jamrules to something more appropriate... someday

The rule subproject serves two purposes, which are very related, but yet distinct:

It identifies subproject relatively to top-level one.
It gives the location of the Jamfile containing it.

 subproject boost/libs/python ;

Note: it could be possible to have two different rules, one which declares locations, and another which declares subproject name. No clear need for such separation is seen so far, but I'm not sure, see comments at the bottom.

So are you proposing that subprojects are named by their location, and nothing else? That would be fine with me if so; I think my previous idea may have been frivolous... but I see from reading on that you are not suggesting that. So, are you suggesting that the argument to "subproject" is to be interpreted only as a path relative to the project root? Maybe we should use a new name, like "path-from-root"

Well, actually, I'd say that using Jamfile locations for naming projects is a little bit too low-level. Concerning subproject my original proposal was to make it at the same time tell location from the project root and implicitly create a subproject. I thought it would be more convenient, but in truth, I'd rather have separate path-from-root (or location, or whatever) rule, with subproject name declared separately. I guess it won't be too hard to use two rules instead of one.

I'm working with the following idea. What do you think?

   project.project ( project-id : requirements * : default-build * )

      Declares a project or subproject. A subproject's id is a path,
      starting with the project id of which it is a subproject. The
      requirements and default build apply to any targets described in
      the Jamfile which do not explicitly declare others. A project
      rule invocation is mandatory in any Jamfile in a project which
      includes subprojects or uses other projects.

   project.jamfile-location( root-to-jamfile )

       Declares the location of this Jamfile with respect to the
       project root, in case the path given in the project rule does
       not describe the location of the jamfile.

   project.source-location( root-to-source )

       Declares that relative paths in this Jamfile are all specified
       relative to the specified directory. Thus, a project with this
       structure:

          root        
          +- build
          |  `- Jamfile
          `- src         
             +- foo.c
             `- bar.c

       might have the following Jamfile:

          project.project foobar ;
          project.jamfile-location build
          project.source-location src ;

          exe foobar : foo.c bar.c ;

The rule subinclude makes project rooted at a named directory a subproject of the current subproject.

 subinclude libs/regexp ;

Note: subinclude rule can be used at any point of Jamfile.

What does it mean to have a subproject? I am assuming the meaning is something like: "When you build in this directory, also build everything that would be built in this other directory"

Yes, this is one aspect. The other is that subproject will be named relatively to the parent. And here I don't understand how it should work. Options:

Jamfile included contains project rule invocation without a path component. Full project name will be $(including-project-name)/$(parameter-to-project-rule)
Same as previous, but included Jamfile passes full project name to the project rule. In which case, subinclude rules have the only semantics of making all the projects defined in included Jamfile to be build when parent project is build.

I think to support building from the subproject, we need to do #2. I don't think I like this definition of subproject, though. I think the project/subproject relationship should be defined by the following:

A shared Jamrules file which is read into every subproject Jamfile
When building into an alternate directory (i.e. using ALL_LOCATE_TARGET - though we'll choose a better name), the build directories are part of the same directory tree, rooted at a directory whose name is the same as that of the project.
Requirements and default build are inherited from project to subproject.
In other words, I don't think a project should neccessarily be forced to build all of its subprojects. It seems to me that "subinclude" is the wrong name, too - it should be something like "build-project" instead.
It also seems clear to me that even if figure out a way to consistently allow subprojects to be identified by paths which are different from their Jamfile locations, it will be confusing for users. I think we need a separate syntax to distinguish the naming of targets (and projects) from the naming of relative paths. For example:

@boost/python - a project ID
boost/python - a path
boost/python/@libboost_python - a target called libboost_python described in the Jamfile located at boost/python

We may also need a better way to describe paths relative to the project root than $(TOP)/foo/bar... but that's another question.

The rule project has yet another function: it can declare project locations:

 project boost/python : libs/python ;
 project fortune : /home/ghost/build/fortune ;

In the first case, an alias is created. In the second case, it's stated that project "fortune" is located in "/home/ghost/build/fortune", in case that project will be needed.

I don't see what distinguishes the two cases. Also, I'm a little bit uncomfortable with overloading that rule's meaning. It sounds like, in the 1-arg case, you are saying "the name of this project is..." and in the 2-arg case you are saying "the name of some other project is...". Hmm, maybe that's not so bad after all. In fact, is there any reason for a project to name itself? Maybe it's only neccessary for projects to name other projects.

The cases are distinguished by using absolute path name in one case. I'm actually thinking that there's no great difference between those two statements. Both say: "project named such-and-such is located at...". As for need of project to name itself... well, I don't see such need! Moreover, it entirely eliminates the question of dealing with project renaming, raised somewhere below. Yet, how will project aliases be defined? Won't we need some symbolic name to reference the current top-level project?

I can't help but think that things are getting too complicated to justify any convenience gained from project renaming and aliasing. Maybe we should just go with path names to Jamfiles for now?

Note: what to do if a name declared for external project differs from how the project calls itself. Maybe "use/external_project" rule? Can project renaming be usefull?

Possibly, in case of conflicts it could. Or, suppose you have a project that needs to compile with several different versions of another project. One would need a way to represent boost-1.25.0 and boost-1.26.1, so it might be useful in that way. Still, I am afraid that this feature may introduce too much complication at this stage.

Yes, I was thinking about exactly the same possible usage! But guess you're right -- if it won't be possible to support it "automagically" (i.e. due to some other decision), there's no need to bother with it now.

When project structure is defined by those rules, we'll have a tree of projects, where each project refers to some directory. It is possible for several projects to refer to the same directory (project aliasing). It could happend that a project does not refer to any directory -- e.g. the following might be convenient:

 project foo/test/lib1 : src/lib1/test ;
 project foo/test/lib2 : src/lib2/test ;

Here, project foo/test will be implicitly created, with no directory assigned to it.

I don't understand. It looks like you are saying these projects refer to src/lib1/test and src/lib2/test, respectively. What do you mean?

Actually, I'm trying very hard to have a tree of projects as the result of project, subproject and subinclude rule invocations. Might be mistaken, but see no other way to avoid been lost in a maze of project names, project locations and project aliases. So, in this example, I mean to say: "We have foo/test/lib1, then we should have a parent to that project, foo/test, but it's not declared anywhere. Let's declare it implicitly."

Are you sure that's the best strategy? Usually the best way to avoid being lost in a maze is to do more things explicitly.

Note: actually, this is slightly artifical example. Real reason is that I wanted to be able to declare exported targets (see below), without explicitly defining projects they're in.

That done, project reference from some Jamfile is done the way directory search is made (and ".." is treated like the reference for the parent project).

Example, please?

Example 1.
Consider reference to project boost/python. We look at boost project and see what children it has. We see python here. Then we realized that python is the last component of the project name and expect the project to define Jamfile location. (Actually, this depends. In a scheme where all Jamfiles are read, and which won't be used, we can directly look inside corresponding project for targets or whatever.

I don't understand that sentence at all. How do you read all the Jamfiles if you don't know where they are? And what does it mean to "directly look inside the corresponding project for targets?"

When project names are defined by some top-level file, we'd need to include Jamfile.) Just seen another problem: if project names are defined at top-level, subinclude semantics will become tricky -- it will request rebuilding projects that are declared in top-level file as located in the directory that is included. Uph!

That sentence doesn't parse for me either. Sorry, I'm trying, really!

Example 2.
We're in project something/src/asm_estimate and see in some rule <lib>../common. We need to find the project named .., and, naturally, it is the same as the parent project something/src. The target common is looked in that project, then.

Okay, I'm still a bit confused about what you mean by "declare exported targets (see below), without explicitly defining projects they're in", then. This doesn't sound any different from functionality we already have.

Note: It is possible to define an incorrect structure. E.g. top-level Jamfile might contain:

 project everything;
 subinclude magic;
 project everything/magic : magic ;

This should be an error.

Why? I don't see it.

Because in my original scheme, subinclude declare a child project, here everything/magic. Later project rules attempts to declare a project of the same name, which should be error, at least in case when different locations are assigned to the project. Whether it should be error when location is the same, I'm not sure.

Targets naming

There are two sides to target naming in presence of multiple projects: using and declaring.

Using

Syntax to use a target in a different (sub)project is:

 exe test : test.cpp <*>boost/regexp/libboost_regexp ;

Are you just proposing a replacement for the <lib> syntax? That would be fine with me; I never liked it and it's not very general. I think perhaps it would be better to write:
libraries test : boost/regexp/libboost_regexp ;
or something. Another idea: what if we just wrote:
exe test : test.cpp boost/regexp/libboost_regexp ;
In this case, every element would first be assumed to be a target. We could attempt to read the corresponding Jamfile, and if it was found and the named target is contained therein, we know what's being referred to. Otherwise, we assume that it's just a regular source file. Too tricky? Probably, but an interesting thought nonetheless.

Guess we really should replace <lib> with something. I don't like idea with libraries rule as it gets us back to Jam's LinkLibraries and is not generic. Idea with using no prefix is worth considering -- it would improve user interface a lot. If project tree idea is pushed to the end, we'll have actually an alternative directory-like structure. In which case boost/regex/libboost_regexp will be just found via search in that structure. Furhter, using files which do not reside in directories where any subproject is rooted should also be possible, althought, as you say, "tricky". For example, foo/lib1 might be located in foo/src/lib1 and Jamfile might contain:
lib lib1 : lib1.cpp aux/routine.cpp
In that case, precisely as you propose, lookup for "aux" project will be performed, will find nothing and aux/routine.cpp will be used in a regular fashion. The only problem is that, generally speaking, if aux is some project, we'd have to assume the directory where it recides contain routine.cpp file, and use it. What if there's both aux project and aux subdirectory of current dir? We can't detect that sitiation, but I think we shouln't bother.

No, I'm convinced now that our only hope of salvation is being more explicit. We have to give people a simple, clear, recognizable way to say what they mean.

Declaring

When project is managed by Boost.Build, there's no special setup needed to make targets available for other projects. Otherwise, a project should define a special file called "Jam...." which can use the "target" rule:

 rule target ( name : type : exported_requirements * : target_locations * ) 
 {
  ....
 }

Example usage is:

 target wd : bin : : <*>bin/wd ;
 target expat : lib : <include>include : lib/expat.so <debug>lib/expat_d.so

Note: I still don't know exact syntax for exported requirements and target locations.

Is the target rule intended to describe pre-built targets which don't have a Jamfile?

Yes, this is the primary purpose. Once the rule semantics is agreed upon, it can be used for regular projects, just for conveneince (not sure this is needed).

Note: can we avoid having platform-specific names here? Seems like no.

We might be able to avoid it. Suppose instead of "target", we used the ordinary declaration syntax "dll expat ;" with no sources, followed by "prebuilt expat"?

Reasonable option. We'd only need to anticipate the possibility of location/naming scheme different from that used by boost.build, such as -d suffix in my example. But guess this is not a problem.

Hard questions

It appears to me that some top-level file with all project and target rule invocations might be good. Probably, subproject rule is too confusing and should be split?

Split how?

By making one rule which defines Jamfile location (path-from-root/location), and delegating project declaring semantics to project.

[buy lipitor online] [buy lipitor] [[buy lipitor online]]

[buy fioricet online] [buy fioricet] [[buy fioricet online]]

Disclaimer: This site not officially maintained by Boost Developers