Create Komodo Edit package on FreeBSD (need help building from source)

Hi there,

I am trying to update the Komodo Edit package on FreeBSD. Version 6 was previously made to work using the precompiled Linux binaries, using the Linux compatibility layer on FreeBSD. Since Komodo Edit is based on the Mozilla source code, and FreeBSD has a native Firefox package, I’m sure it is possible to get a native Komodo Edit working on FreeBSD.

I just need a little help getting it to build from source…

  1. I extract the Komodo Edit source from GitHub commit 2f96f3e to:
    /KomodoEdit/

  2. Since the FreeBSD package building infrastructure (ports) does not allow network connections once the build process has started, I download the source code for the Firefox 31.3 extended support release here:
    http://ftp.mozilla.org/pub/mozilla.org/firefox/releases/31.3.0esr/source/firefox-31.3.0esr.source.tar.bz2
    I extract this Firefox source code to:
    /KomodoEdit/mozilla/build/moz3100-ko9.10/mozilla/

  3. To satisfy the build process, I create an empty file:
    /KomodoEdit/mozilla/build/moz3100-ko9.10/mozilla/.hgignore

  4. I extract the PyXPCOM source code from:
    http://hg.mozilla.org/pyxpcom/archive/700115e8e3be.tar.bz2
    to:
    /KomodoEdit/mozilla/build/moz3100-ko9.10/mozilla/extensions/python/

  5. I run the commands:

cd /KomodoEdit/mozilla
/usr/local/bin/python2.7 build.py configure -k 9.10 --no-mar --python=/usr/local/bin/python2.7
/usr/local/bin/python2.7 build.py -v clean all

After 8 minutes of compiling, I get the following output:

 8:16.46 YarrCanonicalizeUCS2.o
 8:16.56 YarrInterpreter.o
 8:16.67 YarrPattern.o
 8:16.86 YarrSyntaxChecker.o
 8:27.58 /KomodoEdit/mozilla/build/moz3100-ko9.10/mozilla/config/recurse.mk:39: recipe for target 'compile' failed
 8:27.58 gmake[4]: *** [compile] Error 2
 8:27.58 /KomodoEdit/mozilla/build/moz3100-ko9.10/mozilla/config/rules.mk:592: recipe for target 'default' failed
 8:27.58 gmake[3]: *** [default] Error 2
 8:27.58 /KomodoEdit/mozilla/build/moz3100-ko9.10/mozilla/client.mk:381: recipe for target 'realbuild' failed
 8:27.58 gmake[2]: *** [realbuild] Error 2
 8:27.58 client.mk:168: recipe for target 'build' failed
 8:27.58 gmake[1]: *** [build] Error 2
 8:27.62 37 compiler warnings present.
Traceback (most recent call last):
  File "build.py", line 2946, in <module>
    sys.exit( main(sys.argv) )
  File "build.py", line 2942, in main
    return build(args)
  File "build.py", line 2766, in build
    newArgv = targetFunc(argv)
  File "build.py", line 2453, in target_all
    target_mozilla()
  File "build.py", line 2369, in target_mozilla
    buildDir, log.info)
  File "build.py", line 249, in _run_in_dir
    _run(cmd, logstream=None)
  File "build.py", line 231, in _run
    raise OSError("error running '%s': %r" % (cmd, status))
OSError: error running 'python mach --log-file /KomodoEdit/mozilla/build/moz3100-ko9.10/mozilla/mach.log build ': 2
*** Error code 1

That error output doesn’t tell me much about what actually failed. The Mozilla build FAQ states that the problem can be found by going to the directory in which the build failed (within the obj directory) and type make to relaunch the build for this piece of code, displaying more detailed error messages.

I’m not sure if I am doing this right, but since the last file that was being built was YarrSyntaxChecker.o, I changed into the object directory where I found that file was being built and ran gmake to get more detailed output of the error:

cd /KomodoEdit/mozilla/build/moz3100-ko9.10/mozilla/ko-rel/js/src/
gmake

The final output I get is shown below:

gmake[2]: Entering directory '/KomodoEdit/mozilla/build/moz3100-ko9.10/mozilla/ko-rel/js/src/shell'
js
/KomodoEdit/mozilla/build/moz3100-ko9.10/mozilla/ko-rel/_virtualenv/bin/python /KomodoEdit/mozilla/build/moz3100-ko9.10/mozilla/config/expandlibs_exec.py --depend .deps/js.pp --target js --uselist --  c++ -o js -Qunused-arguments   -I/usr/local/include -Qunused-arguments -Wall -Wpointer-arith -Woverloaded-virtual -Werror=return-type -Werror=int-to-pointer-cast -Wtype-limits -Wempty-body -Werror=conversion-null -Wsign-compare -Wno-invalid-offsetof -Wno-c++0x-extensions -Wno-extended-offsetof -Wno-unknown-warning-option -Wno-return-type-c-linkage -Wno-mismatched-tags -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -fno-math-errno -std=gnu++0x -pipe  -DNDEBUG -DTRIMMED -g -O -fomit-frame-pointer js.o jsheaptools.o jsoptparse.o   -pthread  -Wl,-z,noexecstack -Wl,-z,text    -Wl,-rpath-link,../../../dist/bin -Wl,-rpath-link,/KomodoEdit/mozilla/build/moz3100-ko9.10/mozilla/ko-rel/dist/lib   -L../../../dist/bin -L../../../dist/lib -L/KomodoEdit/mozilla/build/moz3100-ko9.10/mozilla/ko-rel/dist/lib -lnspr4 -lplc4 -lplds4 ../../../js/src/editline/libeditline.a ../libjs_static.a /KomodoEdit/mozilla/build/moz3100-ko9.10/mozilla/ko-rel/modules/zlib/src/libmozz.a -Wl,--whole-archive ../../../dist/lib/libmozglue.a -Wl,--no-whole-archive -rdynamic -lm
Executing: c++ -o js -Qunused-arguments -I/usr/local/include -Qunused-arguments -Wall -Wpointer-arith -Woverloaded-virtual -Werror=return-type -Werror=int-to-pointer-cast -Wtype-limits -Wempty-body -Werror=conversion-null -Wsign-compare -Wno-invalid-offsetof -Wno-c++0x-extensions -Wno-extended-offsetof -Wno-unknown-warning-option -Wno-return-type-c-linkage -Wno-mismatched-tags -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -fno-math-errno -std=gnu++0x -pipe -DNDEBUG -DTRIMMED -g -O -fomit-frame-pointer /KomodoEdit/mozilla/build/moz3100-ko9.10/mozilla/ko-rel/js/src/shell/tmplK9tVv.list -pthread -Wl,-z,noexecstack -Wl,-z,text -Wl,-rpath-link,../../../dist/bin -Wl,-rpath-link,/KomodoEdit/mozilla/build/moz3100-ko9.10/mozilla/ko-rel/dist/lib -L../../../dist/bin -L../../../dist/lib -L/KomodoEdit/mozilla/build/moz3100-ko9.10/mozilla/ko-rel/dist/lib -lnspr4 -lplc4 -lplds4 ../libjs_static.a /KomodoEdit/mozilla/build/moz3100-ko9.10/mozilla/ko-rel/modules/zlib/src/libmozz.a -Wl,--whole-archive ../../../dist/lib/libmozglue.a -Wl,--no-whole-archive -rdynamic -lm
/KomodoEdit/mozilla/build/moz3100-ko9.10/mozilla/ko-rel/js/src/shell/tmplK9tVv.list:
    INPUT("js.o")
    INPUT("jsheaptools.o")
    INPUT("jsoptparse.o")
    INPUT("../editline/editline.o")
    INPUT("../editline/sysunix.o")

c++: error: no such file or directory: '/KomodoEdit/mozilla/build/moz3100-ko9.10/mozilla/ko-rel/modules/zlib/src/libmozz.a'
c++: error: no such file or directory: '../../../dist/lib/libmozglue.a'
/KomodoEdit/mozilla/build/moz3100-ko9.10/mozilla/config/rules.mk:729: recipe for target 'js' failed

Has anyone come across these errors before during compilation? Is this the right way to find the actual build error? How can I debug this further?

Have you checked the contents of /KomodoEdit/mozilla/build/moz3100-ko9.10/mozilla/mach.log ?

That sounds like things are being built out of order; that’s odd, since these days the Mozilla build system should be good enough to support parallel builds. Does anything change if you used build.py configure .... -j1 ? That should force a linear compilation, which is slower but much easier to figure out what’s going wrong.

Another idea - try compiling the firefox 31 source code directly (outside of the Komodo repo) - if that fails, then it’s a mozilla related issue and you may need to file a bug with them, else if the firefox build works then it’s a Komodo build issue and we can help dig down further.

To your point toddw, I have realised that the Firefox source needs a number of patch to build correctly on FreeBSD. This is automatically handled by the FreeBSD ports system. On top of that, the build.py script has a number of issues preventing it from being useful on FreeBSD.

To allow the necessary FreeBSD patches and build processes to be used, I have designed my draft komodo-edit port to not use build.py at all, but instead have replicated the steps it takes (plus the necessary steps for FreeBSD).

I have it mostly working, but realised I have missed a step to preprocess the file in mozilla/patches-new/komodo-31.0/ before moving them into the Mozilla source directory.

Can someone please explain how I can call the preprocessor directly from the command line, and how to pass it any config necessary?

Just saying I’m a Komodo IDE user, and I would love to have a native FreeBSD package. Good to know I am not the only one here!