make/netbeans/README

Working on OpenJDK using NetBeans
    This note describes how to work on the OpenJDK from NetBeans. We've
    provided several NetBeans projects as starting points. Below we'll
    describe how to use them, as well as how to create your own.

Getting Started
    In addition to the source bundle for Open JDK, you'll need to download
    and install copies of the JDK and of NetBeans 6. And if you want to run
    tests on the JDK (you do want to run tests, right?), you'll need to
    install the jtreg test harness.

    In this note, when pathnames are not fully specified, they should be
    interpreted as being relative to the directory containing this README
    and the NetBeans projects themselves.

        The JDK build process is largely make-based, and is not
        exceptionally tolerant of pathnames with spaces in them (such as
        "Program Files". Please be sure to install everything in a
        directories whose paths don't have any spaces!

    Downloading the JDK
        You've probably done this a million times. Download and install it
        from http://java.sun.com/javase

    Downloading the OpenJDK sources
        Since you're reading this, d you've already downloaded the OpenJDK
        source bundle.  Later in this document we'll refer to the location
        where you installed the Open JDK sources as *install-dir*.

    Downloading a pre-built, JDK 7
        This will be necessary to do builds of some of the projects.  In
        general, you want to download and install a pre-built JDK that
        corresponds to the OpenJDK sources you download.  Building the entire
        OpenJDK depends on a few parts of the pre-built JDK.  Get this from
        http://download.java.net/jdk7/binaries

        Note: For working on certain projects, like JMX and JConsole, you 
              may find convenient to use a pre-built version of JDK 7 (or 
              OpenJDK) rather than building your own. This will allow you
              to build only that part of the OpenJDK sources which correspond
              to that project. 

    NetBeans 6
        Yep, NetBeans *6*. Nope, not FCS'd yet. We're on the edge here,
        enjoy it! Get the latest working development build of NetBeans 6
        from http://netbeans.org

    jtreg
        "jtreg" is the test harness for running OpenJDK's regression tests.
        Get it from http://openjdk.java.net/jtreg

    Ant
       NetBeans comes with ant, but if you use a separately-installed copy
       please make sure that it is at least version 1.7.0.

Configuring
    Building OpenJDK is hard and complex. No, strike that. While it's not
    exactly "easy", we've got it down to *relatively* small set of
    properties you need to set.

    The NetBeans projects provided here share a fair amount of common
    structure. They share properties values where it makes sense. Each
    project loads properties from these properties files, in this order

        ${basedir}/nbproject/private/build.properties
        $HOME/.openjdk/${ant.project.name}-build.properties
        $HOME/.openjdk/build.properties
        ${basedir}/build.properties

    (${basedir} refers to the directory containing a particular NetBeans
    project.) The first time a property defined determines value: it is
    *not* overridden if it is read from properties files read later. The net
    result is that by carefully choosing where to define a property, you can
    have it for a specific project, all uses of a specific project (useful
    if you work on multiple copies of the OpenJDK sources), all projects, or
    only projects in a specific sandbox.

    With that in mind, please set the following properties. Presuming you
    want the same values for all your work, set them in
    $HOME/.openjdk/build.properties.

    * bootstrap.jdk
        Set to the location where you installed JDK 7.

    * jtreg.home
        Set to the location where you installed jtreg.

    * make.options
        Some of the projects invoke "make", since they compile native code.
        The make.options property is for passing information about what you
        installed where to make.  Change the paths to fit your particular
        situation:

        make.options=\
            ALT_BOOTDIR=/home/me/bin/jdk1.6.0 \
            ALT_JDK_IMPORT_PATH=/home/me/bin/jdk1.7.0 \
            OPENJDK=true

        The trailing '\' are important, so that make gets the above as a
        single set of options.

        You might want to add additional additional options: see the README
        for the project you're using for more information.  And see
                 *install-dir*/jdk/make/README-builds.html
        to read much more about building the JDK.

  Windows-specific configuration
    First, please note that the entire JDK cannot currently be built on
    Windows platforms.  This will likely limit your ability to build
    make-based projects.  See 
         *install-dir*/jdk/make/README-builds.html
    for full information on issues with building on the Windows platform.

    That said, there are two ways to work with the Windows-required settings
    for the Microsoft tools. Either:

    * Set environment variables values in Windows
        Doing so means accessing the System control panel in Windows, and
        setting the environment variables there.

        By doing so, you can launch NetBeans by double-clicking its icon,
        and the environment variable values will be available.

    * Set environment variable values in a shell
        Doing so means adding the settings to an init file (e.g. .bashrc,
        .cshrc, etc.) or a file that you source before running NetBeans. In
        this case, you'll have to launch NetBeans from the command line in a
        shell in which you've set the environment variables.

    In either case, the end result should be that the settings are available
    to the make-based build process when it runs from within NetBeans.

    The make-based builds presumes that you're using cygwin, and expects to
    find "make" in c:\cygwin\bin\make. If you've installed cygwin elsewhere,
    set "make" in a properties file.

  Configuring Project Properties
    A note of caution is in order: These are NetBeans *freeform* projects.
    If you use the NetBeans GUI to examine them, things are likely to not
    look "right". Please don't edit them there, please instead use a text
    editor.

  Locale Requirements
    To build the Open JDK sources, be certain that you are using the "C"
    locale on Unix (R) platforms, or "English (United States)" locale on
    Windows.

Platforms and architectures, oh my!
    The Open JDK can be built for a variety of operating system platforms
    and hardware architectures. The resulting builds are always placed in a
    directory which contains the platform and architecture as part of the
    pathname, as in *platform*-*arch*. For example, if you build the jdk
    project on a Linux platform running on x86 hardware, the resulting build
    will be in:

    *install-dir*/jdk/build/linux-i586

    We've provided support for some platforms and architectures in
    common/architectures. Add another, if your needs require it.

Provided NetBeans projects
    This section describes the NetBeans projects that help you work on
    particular parts of the JDK. While they're largely similar in structure
    and should work the way you expect NetBeans projects to work: edit,
    build, test, etc. But there are some differences. They don't all support
    the same targets (e.g., there's nothing to run in jarzip project).

    Some projects are built by invoking make, since they involve compilation
    of native code or other activities that cannot be done by javac. We call
    these "make-based", and call all others "ant-based".

    They all are configured by way of a build.properties file, which
    specifies what subdirectories of the JDK sources they manipulate, what
    directories contain their tests, whether they use make or ant, etc.

    The very first time you open any one of these projects on set of Open
    JDK sources, NetBeans will scan the entire set of sources, not just
    those for the project you opened. This will take a few minutes, but will
    ensure that Go To Type, Go To Source, and so on work as expected. Later,
    when you open other projects on the same Open JDK sources, there will be
    at most a slight delay.

    There's a README accompanying each project. Most are text files, which
    you can Open in NetBeans, some are HTML files, in which case unless you
    enjoy reading raw HTML, you're better off choosing the *View* menu item
    from the context menu, which will display the README in your web
    browser.

    Finally, note that these projects were all created by different people,
    and are while some attempt has been made to make them look and behave
    the same, they are maintained separately and will vary somewhat.

    The projects currently provided are:

    jdk (directory "jdk")
        A convenient starting point for the other projects, and from which
        you can build the entire OpenJDK. Please note that depending on your
        hardware, this could take a *very* long time. The results of the
        build are in *install-dir*/jdk/build/*platform*-*arch*.

    world (directory "world")
        This project builds both the Hotspot VM and all of JavaSE. Please
        note that pretty much regardless of your hardware, this *will* take
        a long time, and use *lots* of disk space (more than 3GB). The
        results of the build are in
        *install-dir*/build/*platform*-*arch* and
        *install-dir*/build/*platform*-*arch*-fastdebug.

        Consult the project's README file for details.

    AWT & Java2d (directory "awt2d")
        For working on AWT and Java2d. Supports running the Font2DTest demo.

        This is a make-based project: In order to build this project, you
        should build the jdk project first, since AWT and Java2d include
        native code.

    JConsole (directory "jconsole")
        For working on JConsole. Creates ../dist/lib/jconsole.jar. Supports
        running and debugging JConsole.

        This ant-based project does *not* require that you build the jdk
        project first, provided that you use a pre-built version of JDK 7. 

    Java (TM) Management Extensions (JMX(TM)) API (directory "jmx")
        For working on JMX source code. Creates ../dist/lib/jmx.jar.

        This ant-based project does *not* require that you build the jdk
        project first, provided that you use a pre-built version of JDK 7. 

    Jar & Zip (directory "jarzip")
        For working on jar & zip. It builds the zip library (including
        native code), the jar library, and the jar tool. Creates an
        executable jar program in ../build/*platform*-*arch*/bin/jar.

        This is a make-based project: In order to build this project, you
        should build the jdk project first, since AWT and Java2d include
        native code.

    Swing (directory "swing")
        For working on Swing. Creates ../dist/lib/swing.jar. Supports
        running and debugging the SampleTree demo.

        This ant-based project does *not* require that you build the jdk
        project first, provided that you use a pre-built version of JDK 7. 

    In addition, there are projects for building the compiler, javadoc,
    and related tools, in the OpenJDK langtools component.  These
    projects are separate from those described here, and have their
    own set of guidelines and conventions. For more details, see the 
    README files in make/netbeans in the OpenJDK langtools component.

Running Tests
    We use the jtreg test harness, described more fully at
    http://openjdk.java.net/jtreg

    The OpenJDK tests are in the default Java package, are public classes,
    and have a "static void main(String[] args)" with which they are
    invoked. Some tests are actually shell scripts, which might compile
    code, etc. jtreg is quite flexible.

    To run tests for a project, use *Test Project* from NetBeans. From the
    command line, you can invoke "ant jtreg" on any individual project's
    build.xml file.

    In either NetBeans of on the command line, jtreg prints summary output
    about the pass/fail nature of each test. An HTML report of the entire
    test run is

    ../build/*platform*-*arch*/jtreg/*ant-project-name*/JTreport/report.html

    In that same JTreport directory are also individual HTML files
    summarizing the test environment, test passes and failures, etc.

    More detail on any individual test is under

    ../build/*platform*-*arch*/jtreg/*ant-project-name*/JTwork.

    For example, details about the awt/Modal/SupportedTest/SupportedTest
    test are under the JTwork directory at the same pathname as the test
    itself in a ".jtr" file. For example:

    ../build/*platform*-*arch*/jtreg/*ant-project-name*/JTwork/awt/Modal/SupportedTest/SupportedTest.jtr

    Sometimes you will see that running jtreg has resulted in a failure.
    This does not always mean that a test has an error in it. Jtreg
    distinguishes between these two cases. There are a number of tests that
    are "ignored", and not run, and these are reported as failures.

    You can run a single test by right clicking on it and choosing *Run
    File* from the context menu. Similarly, you can debug a single test by
    choosing *Debug File*.

Debugging
    Debugging is enabled by default in ant-based projects, as if
    "-g:lines,vars,source" were given. You can alter these settings via
    entries in one of the configuration properties files. For example:

     javac.debug=false
     javac.debuglevel=<debug level options>

    To debug a project or test, use NetBeans in the normal way, with *Debug
    Project* or *Debug File*. Note that not all projects provide a target
    that can be debugged, but tests can be debugged.

Creating Javadoc
    You can create Javadoc for any of the projects: just choose *Generate
    Javadoc for Project* from the NetBeans menu. Your default browser will
    open up, displaying the just-generated javadoc.

    Javadoc gets generated into a separate subdirectory for each project.
    For example, the Jar & Zip project's javadoc gets generated in

    ../build/*platform*-*arch*/jtreg/*ant-project-name*/javadoc/jarzip

Cleaning projects
    Each project can of course be cleaned. Make-based and ant-based projects
    differ a little in what exactly gets cleaned. In both cases, all jtreg
    results and javadoc are removed.

    In ant-based projects, project-specific files as determined by the
    project's build.properties file are removed from the classes and gensrc
    directories that are under ../build/*platform*-*arch*.

    In make-based projects, "make clean" is run in the same directories as
    "make all" is run when building the project.

    Please note that the jdk project is "special" with respect to
    cleaning: in this case, the entire ../build directory is removed.
    Similar for the world project.

Creating your own NetBeans project
    The project's we've provided are hopefully a useful starting point, but
    chances are that you want to work on something else. This section will
    describe how to select an existing project, and then adapt it to your
    needs.

  Considerations
    The first consideration is whether or not the code in which you're
    interested needs anything beyond javac and copying of resources to
    build. If so, then you'll need to create a make-based project. If not,
    an ant-based project is possible. See the project descriptions above to
    learn which are make-based, and which are ant-based.

    The second consideration is to consider the files that you'll need. Each
    project is defined by 3 files:

    * build.xml
        This is the ant build script. For a make-based project, they tend to
        have a target for "make clean" and another for "make all", each of
        which invokes "make-run" in the same set of directories. Take a look
        at jarzip/build.xml for an example.

        For an ant-based project, there might be nothing, with all the work
        done via the declaration of properties in the build.properties file.
        Take a look at jconsole/build.xml for an example, and notice how it
        overrides the -pre-compile and -post-compile targets that are
        defined in common/shared.xml (where they are defined to do nothing).

    * build.properties
        This file defines the directories (and possibly files) that are
        included in and excluded from. Basically, a file is considered to be
        in a project if it is mentioned in the includes list, or is
        contained under a directory mentioned in that list, *unless* it is
        explicitly excluded or is contained under a directory that is
        excluded. Take a look awt2d/build.properties for an example.

    * nbproject/project.xml
        This file defines a project for NetBeans for a "freeform" project.
        Each declares several entity references, which are used later in the
        project. For an example, see javadoc/nbproject/project.xml, which is
        an ant-based project. Compare that with
        jarzip/nbproject/project.xml, which is make-based. Not much
        difference! That's because while the jarzip project is make-based,
        it does not have any platform-specifc native code. Contrast that
        with awt2d/nbproject/project.xml, which does have native code;
        notice that it uses platform-specific entity references.

    In summary, we recommend exploring the given projects, and choosing one
    that most closely suits our needs.

  Example: A project for working on collections
    Let's create a project to work with on the collections classes. There's no native
    code here, so an ant-based project will do. Therefore, the jconsole
    project is a reasonable project to use as a starting point.

   Clone the existing project
    Make a directory for the collections project next to the existing projects:

        % mkdir -p collections/nbproject

    Copy files from the jconsole project:

        % cp jconsole/build.properties collections
        % cp jconsole/build.xml collections
        % cp jconsole/nbproject/project.xml collections/nbproject

   Change the set of files included in the project
    The collections sources are all under one directory, and we want to include
    them all. The same is true of the tests. So edit
    collections/build.properties so that it contains these lines:

        includes=\
            java/util/
        excludes=\
            java/util/Calendar.java,\
            java/util/jar/,\
            java/util/logging/,\
            java/util/prefs/,\
            java/util/regex/,\
            java/util/spi/,\
            java/util/zip/,\
            **/*-XLocales.java.template
        jtreg.tests=\
            java/util/**/*Collection/ \
            java/util/**/*Map/ \
            java/util/**/*Set/ \
            java/util/**/*List/

    Notice the trailing "/" in some of those pathnames: that tells NetBeans to
    treat the path as a directory and include (or exclude) everything beneath
    it in the hierarchy.  Note also how we include java/util, but then exclude
    several directories under that which are not related to collections.

    The build.xml for collections is about as simple as can be. First, change the
    name of the project:

        <project name="collections" default="build" basedir=".">

    Then remove the -pre-compile target from the build.xml.  Change the
    -post-compile target to create collections.jar without any manifest, and
    to only contain the collections-related classes.  The jar task now looks
    like this:

        <jar destfile="${dist.dir}/lib/collections.jar">
            <fileset dir="${classes.dir}">
                <include name="java/util/*.class"/>
                <exclude name="java/util/Calendar*.class"/>
            </fileset>
        </jar>

    Also, change the clean target to remove collections.jar instead of
    jconsole.jar.

    Now edit project.xml file. NetBeans uses an internal name and a
    user-visible name, both of which should be changed:

        <name>Collections</name> <!-- Customized -->

        <property name="name">collections</property> <!-- Customized -->

    Inside of <ide-actions>, you'll see actions defined for "run" and
    "debug". The Open JDK sources don't include any interesting Collections
    demos, but leave these here for now: Chances are you'll find or create
    some collections app of your own, and want to run and or debug it.

    Now, open the Collections project in NetBeans. You'll find that it operates
    just like all the other projects.

    If/when you want to have this project run a collections demo, change the run
    target in collections/build.xml to invoke it in whatever manner is appropriate
    for the app. From NetBeans, you should be able to run and debug the app,
    including setting breakpoints in collections code.

Appendix 1: Customizations
    There are several ways to customize NetBeans projects. These projects
    share a common structure, based on common/shared.xml and
    common/make.xml. Because of that sharing, some mechanisms described
    below apply to most any project.

    Several properties can be user-defined (and several should not be
    user-defined!). There are different properties files read. Some default
    targets can be overridden.

  Property files
    When projects are started, and when when ant runs (whether from NetBeans
    or the command line), these properties files are loaded in the order
    shown:

        ${basedir}/nbproject/private/build.properties
        $HOME/.openjdk/${ant.project.name}-build.properties
        $HOME/.openjdk/build.properties
        ${basedir}/build.properties

    Recall that with ant, once a property is defined, its value cannot be
    changed, so it's "first one wins".

    To set or change a property for all your projects, put the change into
    $HOME/.openjdk/build.properties. This will affect all projects,
    regardless of how many copies of the Open JDK sources you have
    installed.

    Let's say you have 2 copies of the Open JDK sources installed on your
    machine. To set or change a property for only the jconsole projects, but
    for both of them, make the change in
    $HOME/.openjdk/${ant.project.name}-build.properties. If you wanted to
    make the change for only one of them, do it in that project's
    ${basedir}/build.properties or
    ${basedir}/nbproject/private/build.properties.

    Note that the ${basedir}/build.properties file is provided as part of
    the Open JDK sources. If you want to make a change for a particular
    project, you can do so there. To be sure that you don't ever
    accidentally check it in to the Open JDK sources, you might prefer to
    change it in ${basedir}/nbproject/private/build.properties.

  User-definable Properties
    You can provide your own definitions for the properties listed below. We
    don't recommend overriding the definitions of other properties.

    The following two properties should be set before you try to use the
    projects with NetBeans or ant:

    * bootstrap.jdk
        Default: None. Please set this, normally in
        $HOME/.openjdk/build.properties.

    * jtreg.home
        Default: None. Please set this, normally in
        $HOME/.openjdk/build.properties.

    These options are for configuring the behavior of make:

    * use.make
        Default: Not set. Set this, normally in ${basedir}/build.properties,
        for a project which is make-based.

    * make
        Default: The right make for the platform, at the normal location, set
        in *install-dir*/jdk/make/netbeans/common/make.xml

    * make.options
        Default: Empty string. Set this to any options you want to pass to
        make, normally in ${basedir}/build.properties.

    The remaining options are for use at your discretion:

    * javac.options
        Default: -Xlint

    * javac.debug
        Default: true

    * javac.debuglevel
        Default: lines,vars,source

    * javadoc.options
        Default: Empty string.  Some projects will need to set this to
        increase the heap for running javadoc.  For example, see the jconsole
        project.

    * javadoc.packagenames
        Default: "none".  Set this only if your project has packages that
        should be javadoc'd which are outside of those listed in the javadoc
        target's packageset.  See the jconsole project for an example.

    * jtreg.tests
        Default: None. Set this to a list of tests and/or directories
        containing regression tests, normally in
        ${basedir}/build.properties.

    * jtreg.options
        Default: Empty string. See http://openjdk.java.net/jtreg

    * jtreg.vm.options
        Default: Empty string. See http://openjdk.java.net/jtreg

    * jtreg.samevm
        Default: false. See http://openjdk.java.net/jtreg

  User-overridable Targets
    The following targets are provided for your convenience in customizing
    various standard actions of the build process. The default action for
    each one is to do nothing.

    These come in pairs, allowing your scripts to take some action before or
    after a standard action.

    * -pre-init
        Runs before any other initialization has been done.

    * -post-init
        Runs before after all other initialization has been done.

    * -pre-compile
        Runs before compilation, whether via ant or make. Note that in the
        case of make, it is before the -build-make target has run, not after
        each individual make-run has run.

    * -post-compile
        Runs after compilation, whether via ant or make.

    * -pre-jtreg
        Runs before regression tests are run.

    * -post-jtreg
        Runs before after regression tests are run.

    In a make-based project, you should override these targets to do the
    build and clean actions required of your project.

    * -build-make
    * -clean-make

Known Issues
  Tests won't run: waiting for lock
    Occasionally when running tests, there will be a delay, followed by a
    message like this:
        Waiting to lock test result cache for
           /tmp/jdk/build/linux-i586/jtreg/jconsole/JTwork for 20 seconds
    The workaround is to stop the tests, rm -rf the offending jtreg/<project>
    directory by hand, and re-run the tests.

  Can't run nor debug a single test in the JConsole test
    In most projects, you can run a single test by opening it in the editor,
    and choosing Run File from the context menu.  If you try this with the a
    JConsole test, instead you'll see that *all* tests from *all* projects
    are run.  The workaround is to not try to run a single JConsole test.
    Debugging is similarly problematic (both running and debugging use the
    same underlying infrastructure).

    If you do Run File a JConsole tests, you can always stop them by pressing
    the stop button in the NetBeans output window.  But you'll be surprised to
    learn that they are actually still running in the background.  The only
    way out of this situation is to exit NetBeans.  A few more tests will run,
    but after restarting NetBeans things will be OK.

Attribution
    UNIX is a registered trademark in the United States and other countries,
    exclusively licensed through X/Open Company, Ltd.