0. Pre-Checkin Checklist Performed from top of repo, default branch, head: [ ] Is tag 'current' at or near head of 'vendor'? hg heads hg tags [ ] Expected differences between vendor and default? Very little of the original source should be modified. hg diff -rcurrent libpng [ ] Are the 'vendor' and 'default' branch source directories 'libpng' and not 'libpng-'? [ ] Have you reverted/blocked checkin of files modified by configure such as Makefile, config.*, libpng-config, libpng.pc, libpng.vers, libtool, etc.? [ ] Built PIC on *nix platforms? [ ] Examined pngfix and libraries for correct building. * 'stage/bin/pngfix' shows no dynamic linkage to libz: $ ldd stage/bin/pngfix linux-gate.so.1 => (0xf7713000) libm.so.6 => /lib32/libm.so.6 (0xf76d7000) libc.so.6 => /lib32/libc.so.6 (0xf7590000) /lib/ld-linux.so.2 (0xf7714000) * 'strings stage/bin/pngfix | egrep 1\\.2\\.' shows correct libz and libpng: $ strings stage/bin/pngfix | grep 1\\.[0-9]\\. 1.6.8 1.2.8 libpng version 1.6.8 - December 19, 2013 libpng version 1.6.8 - December 19, 2013 deflate 1.2.8 Copyright 1995-2013 Jean-loup Gailly and Mark Adler inflate 1.2.8 Copyright 1995-2013 Mark Adler * Libraries have correct versions: $ strings stage/lib/release/libpng.a | grep 1\\.[0-9]\\. 1.6.8 libpng version 1.6.8 - December 19, 2013 libpng version 1.6.8 - December 19, 2013 1.6.8 1.2.8 1.6.8 1.2.8 1. Introduction Simple build of libpng library from libpng.org. Repo structure *now* mostly follows standard conventions (see section at end). Prior to 1.6.7, repo didn't follow standard convention and each release was disconnected from previous releases. Modifications to branch 'default' were re-implemented on every update. The version jump from 1.5.1 to 1.6.7 included a good number of security and reliability fixes. Version number is also reflected in the library name everywhere (libpng15 -> libpng16). There are PNG files in the distribution which Mercurial does not know how to merge. Add the following to .hg/hgrc in the repo so that PNG files are take from the other branch: [merge-patterns] **.png = internal:other 2. Modifications Essential modifications to libpng distribution. 2.1 Windows Project Files * libpng.vcxproj: o /Zc:wchar_t- already established o Debug|Win32: . WholeProgramOptimization => false . LinkTimeCodeGeneration => Default . ProgramDatabaseFileName => $(OutDir)$(TargetName).pdb o Debug Library|Win32: . WholeProgramOptimization => false . ProgramDatabaseFileName => $(OutDir)$(TargetName).pdb o Release|Win32 . WholeProgramOptimization => false . LinkTimeCodeGeneration => Default . ProgramDatabaseFileName => $(OutDir)$(TargetName).pdb o Release Library|Win32: . WholeProgramOptimization => false . ProgramDatabaseFileName => $(OutDir)$(TargetName).pdb * pnglibconf.vcxproj: o Release|Win32 . LinkTimeCodeGeneration => Default * pngtest.vcxproj: o Release|Win32 . LinkTimeCodeGeneration => Default o Release Library|Win32: . LinkTimeCodeGeneration => Default * pngvalid.vcxproj: o Debug|Win32: . LinkTimeCodeGeneration => Default o Debug Library|Win32: . LinkTimeCodeGeneration => Default o Release|Win32 . LinkTimeCodeGeneration => Default o Release Library|Win32: . LinkTimeCodeGeneration => Default * vstudio.sln: o Delete zlib project references * zlib.props: o ZLibSrcDir points to package area where zlib is installed. 2.2 DLLs and Unit Tests Linux and Mac unit tests are functional. Windows will require some more work. That will include finding the correct zlib.lib, build options for wchar_t, and other things. Not difficult, but it will require some iteration. 2.3 Creaky Mac OSX 10.5 Mac is still building old llqtwebkit (based on the http://bitbucket.org/lindenlab/3p-llqtwebit repo) on 10.5. To use this library with a 10.5 build of llqtwebkit, the -mmacosx-version-min must be kept at 10.5 or __bzero() will be used for memset operations. We'll take the optimistic path for now based on the reborn repo (http://bitbucket.org/monty_linden/llqtwebkit2) which builds on more modern releases. 3. Source Origin 1.6.7: http://prdownloads.sourceforge.net/libpng/libpng-1.6.7.tar.gz?download MD5: ab48db8c1acf75472a48c22649a6d13b 1.6.8: http://downloads.sourceforge.net/project/libpng/libpng16/1.6.8/libpng-1.6.8.tar.gz MD5: 29b7065906e2551508a0d7eacd19174e 4. Products Common: * include/libpng16/png.h * include/libpng16/pngconf.h * include/libpng16/pnglibconf.h Windows: * lib/debug/libpng16.lib * lib/debug/libpng16.idb * lib/debug/libpng16.pdb * lib/release/libpng16.lib * lib/release/libpng16.idb * lib/release/libpng16.pdb Mac OS X: * lib/release/libpng16.a (archive) * lib/release/libpng.a (symlink) * lib/debug/libpng16.a * lib/debug/libpng.a Dylib is possible with some work. Most is already done and waiting in build-cmd.sh. Linux: * lib/release/libpng16.a (archive) * lib/release/libpng.a (symlink) * lib/debug/libpng16.a * lib/debug/libpng.a 5. Consumers/Dependents Packages dependent on libpng which will need attention (autobuild.xml) after changes. This is not authoritative, use appropriate build tools to find all dependents. * SDL * openjpeg (*should* be a dependency) * llqtwebkit * viewer =================================================================== Third-Party Library Repo Structure Introduction We want to have a way to capture local modifications to a third-party open-source project, such as libcurl, without needing write access to their public repository. We want to be able to carry forward such modifications to newer versions of the public project. All this should be independent of the organizational decision as to whether it's even desirable to try to submit our local modifications upstream. Fortunately, the Subversion folks articulated a process years ago that addresses this very requirement. They call it "Vendor Branches." The same tactic, suitably adapted, works with Mercurial too. The essence of the idea is that we capture and tag a particular snapshot of the open-source project. We develop our local modifications to that, and the repository tip incorporates them. But when we want to update to a newer version of the public project, we bring it into the repository in such a way that we can discover the changes from the original snapshot and the new one -- and then have Mercurial apply those deltas to the ''combined'' source. The following material is adapted from http://svnbook.red-bean.com/en/1.1/ch07s05.html, the Red Bean Subversion book, but recast for Mercurial. The Linden source for this material is an internal wiki. There may be superceding documentation on the public wiki when you read this. We recommend searching there for updates to conventions below. And note that each particular library may implement variations of this scheme. General Vendor Branch Management Procedure Managing vendor branches generally works like this. You create a named branch ("vendor") to store the vendor source snapshots. Then you import the third party code into that branch. Your modified branch (named "default") is based on "vendor". You always make your local changes to the default branch. With each new release of the code you are tracking you bring it into the "vendor" branch and merge the changes into "default", resolving whatever conflicts occur between your local changes and the upstream changes. Perhaps an example will help to clarify this algorithm. We'll use a scenario where your development team is creating a calculator program that links against a third-party complex number arithmetic library, libcomplex. We'll construct a repository specifically for our locally-modified version of that library. To begin, we must initialize our repository and create at least one file in our "default" branch. $ hg init ourcomplex $ cd ourcomplex $ touch README.txt $ hg commit README.txt Now we can create the vendor branch and do the import of the first vendor drop. We'll call our vendor branch "vendor", and each successive code drop will be tagged "current". $ hg branch vendor $ tar -xjf ../libcomplex-1.0.tar.bz2 $ mv libcomplex-1.0 libcomplex $ hg addremove $ hg commit -m "1.0 source drop" $ hg tag -r tip current $ hg tag -r current 1.0 We now have the current version of the libcomplex source code in branch "vendor", tagged "current" and in a non-version-specific source code subdirectory ("libcomplex"). Next, we merge it into the default branch. It is in the default branch that we will make our customizations. $ hg update default $ hg merge vendor $ hg commit -m "initial: 1.0" We get to work customizing the libcomplex code. Before we know it, our modified version of libcomplex is now completely integrated into our calculator program. A few weeks later, the developers of libcomplex release a new version of their library, version 1.1, which contains some features and functionality that we really want. We'd like to upgrade to this new version, but without losing the customizations we made to the existing version. What we essentially would like to do is to replace our current baseline version of libcomplex 1.0 with a copy of libcomplex 1.1, and then have Mercurial re-apply the custom modifications we previously made to that library to the new version. But we actually approach the problem from the other direction, applying the changes made to libcomplex between versions 1.0 and 1.1 to our modified copy of it. To perform this upgrade, we update our repository to our vendor branch, and update the "current" tag with the new libcomplex 1.1 source code. We quite literally replace the existing files with the new files, clearing out the whole tree and exploding the libcomplex 1.1 release tarball in its place. The goal here is to make the tip of our vendor branch contain only the libcomplex 1.1 code, and to ensure that all that code is under version control. Oh, and we want to do this with as little version control history disturbance as possible. $ hg update vendor $ rm -rf * $ tar -xjf ../libcomplex-1.1.tar.bz2 $ mv libcomplex-1.1 libcomplex $ hg addremove -s 60 $ # Additional 'hg add' and 'hg rm' commands if needed $ hg commit -m "1.1 source drop" After unpacking the 1.1 tarball, hg status will show files with local modifications as well as, perhaps, some unversioned or missing files. If we did what we were supposed to do, the unversioned files are only those new files introduced in the 1.1 release of libcomplex. The missing files are files that were in 1.0 but not in 1.1. The 'hg addremove' command deals with both, and more: the '-s 60' switch directs Mercurial to compare added files to deleted files, recognizing any file at least 60% similar as a move/rename. For simple or stable libraries, the 'hg addremove' command should be reliable. For more complicated libraries subject to refactoring or large gaps of time between updates (e.g. libcurl), it can get a little lost trying to match files in the old release with files in the new release. Pay attention to the output of the command or better still, do dry runs. Files erroneously moved can be excluded with the '-X' option and then dealt with individually with 'hg add' and 'hg rm' commands after 'hg addremove'. (The readme file in the curl library should document a particularly challenging case.) Finally, once our current working copy contains only the libcomplex 1.1 code, we commit the changes we made to get it looking that way. Our current vendor branch now contains the new vendor drop. We move the 'current' tag to the new version (in the same way we previously tagged the version 1.0 vendor drop), and then merge the differences between the version 1.0 and version 1.1 into our default branch. $ hg tag -f -r tip current $ hg tag -r current 1.1 $ hg update default $ hg merge vendor # resolve all the conflicts between their changes and our changes # if you will have conflicts in .hgtags, simply take *all* lines ... $ hg commit -m "update with 1.1" Any additional work needed to get the merged library working can now be done on the default branch. Revision Tags We don't currently make use of Mercurial tags in the build and release process for 3rd-party libraries. But we would like to establish a convention to document update and release points. The tags we would like to establish are: * 'current' Points to a succession of vendor releases checked into the 'vendor' branch. Will almost always be at or close to branch head. * '' Tag on the 'vendor' branch pointing to a verbatim checkin of a 3rd-party's release. Example: '7.21.1' for a particular version of libcurl we have used. * Release-type tags on the default branch aren't as useful given how Mercurial handles tags and how autobuild works. Schematic of a Third-Party Repository Below is the output of the 'hg glog' command showing a library project going through an initial 1.0 release and an update from the vendor to 1.1. Significant revisions in the repository lifecycle are as follows: 0 Creation of the repo with an initial file. 1 1.0 code drop on branch 'vendor' 4 Merge of 1.0 code onto branch 'default' 5 Modifications to library we wish to keep over time. Released. 6 1.1 code drop on branch 'vendor' 9 Merge of 1.1 code onto branch 'default' 10 Fixes to merge yielding production 1.1 library. Released. @ changeset: 10:888229641f6e | tag: tip | user: Monty Brandenberg | date: Wed Oct 30 13:35:51 2013 -0400 | summary: Work to get 1.1 merge working. Release. | o changeset: 9:925ccdf09f50 |\ parent: 5:83c5775c23dc | | parent: 8:977001a08e48 | | user: Monty Brandenberg | | date: Wed Oct 30 13:35:20 2013 -0400 | | summary: update with 1.1 | | | o changeset: 8:977001a08e48 | | branch: vendor | | user: Monty Brandenberg | | date: Wed Oct 30 13:33:49 2013 -0400 | | summary: Added tag 1.1 for changeset 5f6cb89add91 | | | o changeset: 7:59bce0f6d12f | | branch: vendor | | user: Monty Brandenberg | | date: Wed Oct 30 13:33:41 2013 -0400 | | summary: Added tag current for changeset 5f6cb89add91 | | | o changeset: 6:5f6cb89add91 | | branch: vendor | | tag: current | | tag: 1.1 | | parent: 3:8525ad934ecd | | user: Monty Brandenberg | | date: Wed Oct 30 13:33:29 2013 -0400 | | summary: 1.1 source drop | | o | changeset: 5:83c5775c23dc | | tag: 1.0 | | user: Monty Brandenberg | | date: Wed Oct 30 13:32:31 2013 -0400 | | summary: Linden-specific changes to the library. Release | | o | changeset: 4:bccb736585f4 |\| parent: 0:400e4516c406 | | parent: 3:8525ad934ecd | | user: Monty Brandenberg | | date: Wed Oct 30 13:31:40 2013 -0400 | | summary: initial: 1.0 | | | o changeset: 3:8525ad934ecd | | branch: vendor | | user: Monty Brandenberg | | date: Wed Oct 30 13:30:21 2013 -0400 | | summary: Added tag 1.0 for changeset 8ac3828d03bb | | | o changeset: 2:7aa1a1cb62d9 | | branch: vendor | | user: Monty Brandenberg | | date: Wed Oct 30 13:30:14 2013 -0400 | | summary: Added tag current for changeset 8ac3828d03bb | | | o changeset: 1:8ac3828d03bb |/ branch: vendor | tag: 1.0 | user: Monty Brandenberg | date: Wed Oct 30 13:30:09 2013 -0400 | summary: 1.0 source drop | o changeset: 0:400e4516c406 user: Monty Brandenberg date: Wed Oct 30 13:29:16 2013 -0400 summary: Created repo with initial readme file