kernel-autobuild - changelog

• built w/ kernel-autobuild 0.9.4 (dirtyc0w fix)

• built w/ kernel-autobuild 0.9.4 (dirtyc0w fix)

• built w/ kernel-autobuild 0.9.4 (dirtyc0w fix)

• howto
• needed to build kernel 3.18 - otherwise no initramdisk is built on installation

• built w/ kernel-autobuild 0.9.4 (dirtyc0w fix)

• stuff for debian 8
  • kab_run_tests
    • accept fdisk version 2.25.2
    • check for /empty/ kvm.pid and abort
  • kernel-autobuild-prepare-image
    • doopt() - text formatting
    • added debugging output
    • wait for /dev/mapper to finally create fs-entries after kpartx
    • only remove existings loop devices
  • kernel-autobuild-run-tests
    • check for alternative ifup script /etc/qemu-ifup
    • use it eventually
    • remove ”-nographic” option from kvm call (still logs & works)

• changed –dotconfig-setting from …UEVENT_HELPER_PATH'\”\”'… to …UEVENT_HELPER_PATH='\”\”'

• added CONFIG_CRYPTO_TEST=m to –dotconfig-setting resulting in:

  "EXT4_FS=y,EXT4_ENCRYPTION=y,UEVENT_HELPER_PATH'\"\"',CONFIG_SYSTEM_TRUSTED_KEYRING=y,CONFIG_ASN1=m,CONFIG_CRYPTO_USER=m,CONFIG_CRYPTO_MCRYPTD=m,CONFIG_CRYPTO_USER_API_RNG=m,CONFIG_CRYPTO_HASH_INFO=y,CONFIG_ASYMMETRIC_KEY_TYPE=m,CONFIG_ASYMMETRIC_PUBLIC_KEY_SUBTYPE=m,CONFIG_PUBLIC_KEY_ALGO_RSA=m,CONFIG_X509_CERTIFICATE_PARSER=m,CONFIG_PKCS7_MESSAGE_PARSER=m,CONFIG_PKCS7_TEST_KEY=m,CONFIG_CLZ_TAB=y,CONFIG_MPILIB=m,CONFIG_CRYPTO_TEST=m"

• changed –dotconfig-setting from

  "EXT4_FS=y,EXT4_ENCRYPTION=y,UEVENT_HELPER_PATH'\"\"'"

• to

  "EXT4_FS=y,EXT4_ENCRYPTION=y,UEVENT_HELPER_PATH'\"\"',CONFIG_SYSTEM_TRUSTED_KEYRING=y,CONFIG_ASN1=m,CONFIG_CRYPTO_USER=m,CONFIG_CRYPTO_MCRYPTD=m,CONFIG_CRYPTO_USER_API_RNG=m,CONFIG_CRYPTO_HASH_INFO=y,CONFIG_ASYMMETRIC_KEY_TYPE=m,CONFIG_ASYMMETRIC_PUBLIC_KEY_SUBTYPE=m,CONFIG_PUBLIC_KEY_ALGO_RSA=m,CONFIG_X509_CERTIFICATE_PARSER=m,CONFIG_PKCS7_MESSAGE_PARSER=m,CONFIG_PKCS7_TEST_KEY=m,CONFIG_CLZ_TAB=y,CONFIG_MPILIB=m"

• cd /var/lib/kernel-autobuild/4.1/4.1.26–2.3.8.4–1/build/linux-4.1.26
• cp -ax .config .config-4.1.26-1
• make menuconfig

<M>   Userspace cryptographic algorithm configuration  - CONFIG_CRYPTO_USER

<M>   Software async multi-buffer crypto daemon  - CONFIG_CRYPTO_MCRYPTD
<M>   User-space interface for random number generator algorithms - CONFIG_CRYPTO_USER_API_RNG
<M>   Asymmetric (public-key cryptographic) key type  - CONFIG_ASYMMETRIC_KEY_TYPE
--- Asymmetric (public-key cryptographic) key type
 <M>   Asymmetric public-key crypto algorithm subtype - CONFIG_ASYMMETRIC_PUBLIC_KEY_SUBTYPE
 -M-   RSA public-key algorithm                       - CONFIG_PUBLIC_KEY_ALGO_RSA
 <M>   X.509 certificate parser                       - CONFIG_X509_CERTIFICATE_PARSER
 <M>     PKCS#7 message parser                        - CONFIG_PKCS7_MESSAGE_PARSER
 <M>       PKCS#7 testing key type                    - CONFIG_PKCS7_TEST_KEY

• diff .config-4.1.26-1 .config

236c236
< # CONFIG_SYSTEM_TRUSTED_KEYRING is not set
---
> CONFIG_SYSTEM_TRUSTED_KEYRING=y
365a366
> CONFIG_ASN1=m
6940c6941
< # CONFIG_CRYPTO_USER is not set
---
> CONFIG_CRYPTO_USER=m
6947c6948
< # CONFIG_CRYPTO_MCRYPTD is not set
---
> CONFIG_CRYPTO_MCRYPTD=m
7066c7067,7068
< # CONFIG_CRYPTO_USER_API_RNG is not set
---
> CONFIG_CRYPTO_USER_API_RNG=m
> CONFIG_CRYPTO_HASH_INFO=y
7076c7078,7083
< # CONFIG_ASYMMETRIC_KEY_TYPE is not set
---
> CONFIG_ASYMMETRIC_KEY_TYPE=m
> CONFIG_ASYMMETRIC_PUBLIC_KEY_SUBTYPE=m
> CONFIG_PUBLIC_KEY_ALGO_RSA=m
> CONFIG_X509_CERTIFICATE_PARSER=m
> CONFIG_PKCS7_MESSAGE_PARSER=m
> CONFIG_PKCS7_TEST_KEY=m
7176a7184
> CONFIG_CLZ_TAB=y
7178a7187
> CONFIG_MPILIB=m

• built w/ kernel-autobuild 0.9.3
• changed –dotconfig-setting from

  "EXT4_FS=y,EXT4_ENCRYPTION=y"

• to

  "EXT4_FS=y,EXT4_ENCRYPTION=y,UEVENT_HELPER_PATH'\"\"'"

• built w/ kernel-autobuild 0.9.2.1

• kab_run_tests
  • kill kvm on “Kernel panic”
• log Kernel panic
• count and display minutes while testing

• kernel-autobuild
  • variable SOURCE_IMG is transmitted to kab_run_tests

• /usr/share/kernel-autobuild/linux-image-4.1-vserver-amd64 added

• kab_identify_kernel_source_version -
• adopt to changed kernel.org web page content

• changes for kernel 4.1.5
• kernel-autobuild
  • - option –vspatch-url-template added
  • bug fix: if PARAM_PATCHES consists of a single white space it's handled like ””
• global_static
  • set default for SOURCE_IMG
• kab_downloads
  • added comments
  • bug fix: if PARAM_PATCHES consists of a single white space it's handled like ””
• kab_identify_kernel_source_version
  • adopt to kernel.org enforcing https
  • adopt to changed kernel.org web page layout
• kab_identify_vserver_patch
  • handle –vspatch-url-template parameter
• kab_parsecfg
  • set default for VSPATCH_URL_TEMPLATE
  • handle –vspatch-url-template
• kab_run_tests
  • renamed test template image squeeze-kerneltest-…img → kerneltest-…img
  • mv partition from sect 63 to sect 2048 if necessary inside test image, so grub2 would work
• kernel-autobuild-run-tests
  • changed kvm named from “squeeze-kerneltest” to “kernel-autobuild-kerneltest”

Problem:
automatic testing fails - because of errors in the testing routine / kvm image 

• kab_identify_kernel_config_file – fixed error which inhibited proper inheriting of .config data

• built w/ kernel-autobuild 0.8.1.3

• fixed bug which resulted in the wrong package name “lihas-firmware-image” instead of “lihas-firmware-image-3.10” :
  • global_static - added missing ” ”
• global_static – removed senseless comments
• global_static – added ”–template” to usage
• global_static – fix handling for unknown command line elements

• built w/ kernel-autobuild 0.8.1

• you may use build templates now with –template
• new command line option –template
• kab_parsecfg – new config file & command line parser
• global_static – now uses kab_parsecfg
• xml config files which may contain build templates (.conf are deprecated now)
• automagically created “typescript” protocol (see: man script) typescript-YYYYMMDD-HH:MM:SS
• added dependency libxml-application-config for XML::Application::Config
  • get it from deb http://ftp.lihas.de/debian stable main
• added dependency libxml-xpath-perl for XML::XPath
• kab_apply_dotconfigsettings - fixed empty return value

• built w/ kernel-autobuild 0.7.1

• .plan:
  • easier control of new options which may be changed using –dotconfig-setting and –dotconfig-setting-file
  • easier way to patch the vserver patch using –vspatch-patch

• global_static – added option –prompt-before-upload
• kernel-autobuild – added optinal prompting before uploading packets
• $VSPATCHPATCHES deployed:
• kernel-autobuild - provide $VSPATCHPATCHES to kab_downloads, kab_apply_vserver_patch
• kab_downloads - copy $VSPATCHPATCHES to sources
• kab_apply_vserver_patch - expire old VSPATCHPATCHES from SRC_DIR if new ones are required
• kab_apply_vserver_patch - apply $VSPATCHPATCHES to vserver patch
• debian/control - removed quoting on Homepage:
• kab_apply_dotconfigsettings - create $SRC_DIR/dotconfig.changelog from $LOG_DIR/dotconfigmod.log for debian/changelog
• patch-scripts-package-builddeb-v3.10.35+.diff - added optional inclusion of a file indicated by $KAB_CHANGELOG
• kernel-autobuild - assign KAB_CHANGELOG to kab_compile_and_package
• kab_compile_and_package - export KAB_CHANGELOG
• kab_modify_dotconfig - besserer debug-output
• kab_apply_dotconfigsettings - debug-output ins logfile
• Makefile - fixed quoting for DEBIAN_FULL_NAME, DEBIAN_EMAIL, DEBIAN_HOMEPAGE
• kab_unpack_kernel_source - remove build/linux-X.XX.XX-tree before unpacking kernel source
• global_static
• added –dotconfig-setting, –dotconfig-setting-file
  • VSPATCHPATCHES
  • DOTCONFIG_SETTING
  • DOTCONFIG_SETTING_FILE
• implemented –dotconfig-setting, –dotconfig-setting-file –vspatch-patch
• added kab_apply_dotconfigsettings for handling .config mods
• added kab_modify_dotconfig for actually modifying .config
• added 'perl','tee' to executables which are probed for
• added 'coreutils' to dependencies (f. 'tee')
• added kab_apply_dotconfigsettings to kernel-autobuild

• As of 3.4.86–vs2.3.3.9–1 the settings are those of 3.4.75 again, except for CONFIG_NET_DMA=n.
• As of 3.4.86–vs2.3.3.9–2 comes with:

• CONFIG_CGROUP_MEM_RES_CTLR_SWAP_ENABLED - n
• CONFIG_CGROUP_MEM_RES_CTLR_SWAP n y
• CONFIG_SENSORS_MAX16064 n m
• CONFIG_SENSORS_LTC2978 n m
• CONFIG_SENSORS_APDS990X n m
• CONFIG_SENSORS_MAX8688 n m
• CONFIG_SENSORS_ZL6100 n m
• CONFIG_SENSORS_UCD9000 n m
• CONFIG_SENSORS_UCD9200 n m
• CONFIG_SENSORS_BH1770 n m
• CONFIG_SENSORS_ADM1275 n m
• CONFIG_SENSORS_BH1780 n m
• CONFIG_SENSORS_LM25066 n m
• CONFIG_SENSORS_MAX34440 n m
• CONFIG_SENSORS_LIS3_I2C n m

• I tried to set “CONFIG_NET_DMA=y” but it just failed to survive “make oldconfig”.

• the first kernel with the revised build process from kernel-autobuild 07-devel9
• also a patch was needed to patch the vserver patch first (see VS ML)
• here was also a small patch to the builddeb script

• patch-3.10.15-vs2.3.6.6.diff produced some errors
• patch-3.10.15-17-vs2.3.6.6.diff provided by Corey Wright fixed these
• generated patch-3.10.17-vs2.3.6.6.diff:

	tar -xvJf ../source/linux-3.10.17.tar.xz
	cp -ax linux-3.10.17 linux-3.10.17-vs2.3.6.6
	cd linux-3.10.17-vs2.3.6.6
	patch -p1 < ../../source/patch-3.10.15-vs2.3.6.6.diff
	patch -p1 < ../../source/patch-3.10.15-17-vs2.3.6.6.diff
	cd ..
	diff -urNpd linux-3.10.17 linux-3.10.17-vs2.3.6.6 > ../source/patch-3.10.17-vs2.3.6.6.diff
	rm -rf linux-3.10.17 linux-3.10.17-vs2.3.6.6

• build kernel without downloads

  kernel-autobuild --kernel-base-version 3.10 --add-patch /var/lib/kernel-autobuild/3.10/00-resources/patch-scripts-package-builddeb-v3.7+.diff --no-download   --ksrcver 3.10.17 --pver vs2.3.6.6

kernel-autobuild (0.6.2) unstable; urgency=low

• kernel < 3.7.0 has no ipv6 nat - so skip this test in “internal-tests”

kernel-autobuild (0.6.1) unstable; urgency=low

• fix: error conditions from the iptables tests did noch reach kernel_autobuild

• 3.4.61 vs2.3.3.9 uploaded
• 3.4.60 vs2.3.3.9 uploaded
• now things should be up to date again.

• kernel 3.10.9 (lihas2), 3.10.10 (lihas2), 3.10.11 uploaded
• kernel 3.10.9 (lihas1), 3.10.10 (lihas1) ware malconfigured lacking NAT, LOG and some other features.

kernel-autobuild (0.6) unstable; urgency=low

• dependency “ncurses-dev” added
• kab_identify_vserver_patch returns PURL=””, KURL=””, PKVER=”” if given ”–pver”
• added patch-scripts-package-builddeb-v3.7+.diff so kernels 3.(>=7) don't remove linux-firmware-image package
• added “| grub-pc (>= 1.99)” to Suggests:
• fix: global_static: set defaults /before/ reading /etc/kernel-autobuild.conf
• added ”–provider-tag” to global_static
• call kernel make with “KDEB_VERSION” to get version names /including/ the extra-version (PROVIDER_TAG)
• added checks for iptables/v6/nat/log-target into internal-tests
• changes dependencies so kernels >= 3.7 won't conflict with firmware package any more
• package upload (kab_upload_packages) now uploads all .deb in one http connection, so only one repository update has to be done
• sometimes the .config-file template gets lost/confused so subsequently NAT, LOG_TARGET, etc vanish. Added tests to internal-tests so this would be detected at least.
• added support f. linux-image-3.9-amd64, linux-image-3.9-vserver-amd64, linux-image-3.10-vserver-amd64
• added diff f. patching scripts/packages/builddeb on kernel 3.4
• added diff f. patching scripts/packages/builddeb on kernel 3.(>=7)
• added sample .config files f. different kernel versions

• kernel 3.10.x now available

• issues with fdisk (changed behaviour 2.17.2 to 2.20.1) fixed in kab_run_tests

• we are upgrading our build-platform to debian 7 wheezy
• last kernel built on squeeze: 3.4.59–vs2.3.3.9–1
• kernels are to be built on wheezy now
• first kernel built on wheezy: 3.4.59–vs2.3.3.9–2
  • problem: fdisk (util-linux 2.20.1) is so screwed up now one can't re-create a partition starting on block 63 anymore.
• so, wheezy build delayed for now …

• changed dependency of image package from “module-init-tools” to “module-init-tools | kmod” to meet wheezy's needs (inside patch-scripts-package-builddeb.diff)
• broken: you can't control which part of the process is run , so only complete runs are available right now.

kernel-autobuild (0.4) unstable; urgency=low

• broke up monolithic code into modules. new files are:
  • global_static
  • kab_identify_kernel_config_file
  • kab_identify_kernel_source_version
  • kab_unpack_kernel_source
  • kab_apply_vserver_patch
  • kab_identify_vserver_patch
  • kab_apply_post_vserver_patch_patches
  • global_static
  • kab_compile_and_package
  • kab_run_tests
  • kab_build_meta_package
  • kab_upload_packages
  • kab_downloads
• changed dependency of image package from “module-init-tools” to “module-init-tools | kmod” to meet wheezy's needs (inside patch-scripts-package-builddeb.diff)

• kernel “NAME” fuckup:
  • There is a NAME parameter inside the master Makefile
• this would be:

^ kernel version ^ NAME ^ ^

extract_name () {
	MAKEFILE=$( tar -tvJf $1 | grep -e "linux-3.[0-9.]*/Makefile" | awk '{print $NF}' )
	NAME=$( tar -xvJf $1 "$MAKEFILE" -O | grep NAME )
	echo $NAME 
}

• after some fixes, kernel-autobuild produces and tests now 3.9 series vserver kernels

• appliance of patches given with –add-patch put in right place

• vserver feature parameter fixed

• architecture and vserver feature parameterized

• added switch ”–no-unpack” to prevent unpacking the kernel source archive
• added switch ”–no-vspatch” to prevent applying the linux vserver patch to kernel source
• print %KERNEL_AUTO_BUILD::DOWNLOAD::INFO even if download-only is unset

• added switch ”–no-upload” to prevent uploading to apt repository

• 3.4.41 … 3.4.45 have been skipped. I wanted to fix the dependency issue before uploading new kernels which took a while.

• patch scripts/packages/builddeb inside kernel source tree, so dependencies get fixed:
  • (meta-)package “linux-image-3.4-vserver-amd64”
    • Depends: linux-firmware-image-3.4 (= 3.4.40-vs2.3.3.9+lihas4-1), linux-image-3.4.40-vs2.3.3.9+lihas4 (= 3.4.40-vs2.3.3.9+lihas4-1)
  • package “linux-firmware-image”
    • renamed to “linux-firmware-image-3.4”
    • Conflicts: linux-firmware-image
    • Replaces: linux-firmware-image
    • Provides: linux-firmware-image
  • package “linux-image-3.4.40-vs2.3.3.9+lihas4”
    • Depends: module-init-tools, linux-base (>= 2.6.32-31), initramfs-tools (>= 0.55) | linux-initramfs-tool
    • Recommends: linux-firmware-image-3.4 (>= 3.4.40-vs2.3.3.9+lihas4), util-vserver
    • Suggests: linux-doc-2.6.32, grub | lilo (>= 1:22.8-8.2~)
    • Breaks: initramfs-tools (« 0.55), lilo (« 1:22.8-8.2~)
    • Provides: linux-image, linux-image-3.4, linux-modules-3.4.40-vs2.3.3.9+lihas4-amd64
  • package “linux-headers-3.4.40-vs2.3.3.9+lihas4”
    • Provides: linux-headers, linux-headers-3.4
  • package “linux-libc-dev”
    • renamed to “linux-libc-dev-3.4”
    • Provides: linux-headers, linux-headers-3.4
    • Conflicts: linux-kernel-headers
    • Replaces: linux-kernel-headers
    • Provides: linux-kernel-headers
  • patch is: /var/lib/kernel-autobuild/3.4/00-resources/patch-scripts-package-builddeb.diff

<M>     UDP: socket monitoring interface

<M> Layer Two Tunneling Protocol (L2TP)
--- Layer Two Tunneling Protocol (L2TP)
  <M>   L2TP debugfs support
  [*]   L2TPv3 support (EXPERIMENTAL)
  <M>     L2TP IP encapsulation for L2TPv3
  <M>     L2TP ethernet pseudowire support for L2TPv3
<M>   6lowpan support over IEEE 802.15.4
--- Network packet filtering framework (Netfilter)
<M>   IP set support

  <M> Netfilter NFACCT over NFNETLINK interface
  [*]   Connection tracking zones
[*]   Connection tracking events
[*]   Connection tracking timeout
[*]   Connection tracking timestamping
  <M>   SNMP service protocol support
  <M>   Connection tracking timeout tuning via Netlink
<M>   set target and match support
<M>   AUDIT target support
<M>   CHECKSUM target support
<M>   "CT" target support
<M>   IDLETIMER target suppor
<M>   LOG target support
<M>   "TEE" - packet cloning to alternate destination
<M>   "addrtype" address type match support
<M>   "cpu" match support
<M>   "devgroup" match support
<M>   "ipvs" match support
<M>   "nfacct" match support

<M> DMI table support in sysfs

> CONFIG_INET_UDP_DIAG=m
> CONFIG_NETFILTER_NETLINK_ACCT=m
> CONFIG_NF_CONNTRACK_ZONES=y
> CONFIG_NF_CONNTRACK_TIMEOUT=y
> CONFIG_NF_CONNTRACK_TIMESTAMP=y
> CONFIG_NF_CONNTRACK_SNMP=m
> CONFIG_NF_CT_NETLINK_TIMEOUT=m
> CONFIG_NETFILTER_XT_SET=m
> CONFIG_NETFILTER_XT_TARGET_AUDIT=m
> CONFIG_NETFILTER_XT_TARGET_CHECKSUM=m
> CONFIG_NETFILTER_XT_TARGET_CT=m
> CONFIG_NETFILTER_XT_TARGET_IDLETIMER=m
> CONFIG_NETFILTER_XT_TARGET_LOG=m
> CONFIG_NETFILTER_XT_TARGET_TEE=m
> CONFIG_NETFILTER_XT_MATCH_ADDRTYPE=m
> CONFIG_NETFILTER_XT_MATCH_CPU=m
> CONFIG_NETFILTER_XT_MATCH_DEVGROUP=m
> CONFIG_NETFILTER_XT_MATCH_IPVS=m
> CONFIG_NETFILTER_XT_MATCH_NFACCT=m
> CONFIG_IP_SET=m
> CONFIG_IP_SET_MAX=256
> # CONFIG_IP_SET_BITMAP_IP is not set
> # CONFIG_IP_SET_BITMAP_IPMAC is not set
> # CONFIG_IP_SET_BITMAP_PORT is not set
> # CONFIG_IP_SET_HASH_IP is not set
> # CONFIG_IP_SET_HASH_IPPORT is not set
> # CONFIG_IP_SET_HASH_IPPORTIP is not set
> # CONFIG_IP_SET_HASH_IPPORTNET is not set
> # CONFIG_IP_SET_HASH_NET is not set
> # CONFIG_IP_SET_HASH_NETPORT is not set
> # CONFIG_IP_SET_HASH_NETIFACE is not set
> # CONFIG_IP_SET_LIST_SET is not set
> CONFIG_NF_NAT_SNMP_BASIC=m
> CONFIG_L2TP=m
> CONFIG_L2TP_DEBUGFS=m
> CONFIG_L2TP_V3=y
> CONFIG_L2TP_IP=m
> CONFIG_L2TP_ETH=m
> CONFIG_IEEE802154_6LOWPAN=m
> # CONFIG_PPPOL2TP is not set
> CONFIG_DMI_SYSFS=m

repository speedup:
packets 3.4.11 .. 3.4.22 moved to lihas-kernel/packages-archive. reduced directory size from 6.6BG to 740MB

• –kernel-base-version added
• /usr/share/kernel-autobuild/linux-image-3.7-vserver-amd64 added
• so, one may run “kernel-autobuild –kernel-base-version 3.7” now

kernel.org changed page layout - xpaths changed in kernel-autobuild

vserver patch failed:

  patching file net/ipv4/inet_diag.c
  ...
  Hunk #8 FAILED at 691.
  ...
  1 out of 11 hunks FAILED -- saving rejects to file net/ipv4/inet_diag.c.rej

--- net/ipv4/inet_diag.c        2012-12-18 14:01:14.000000000 +0000
+++ net/ipv4/inet_diag.c        2012-11-18 23:52:30.000000000 +0000
@@ -691,6 +699,7 @@
                                continue;
 
                        if (bc) {
+                               /* TODO: lback */
                                entry.saddr =
 #if IS_ENABLED(CONFIG_IPV6)
                                        (entry.family == AF_INET6) ?

action:

• removed this chunk from vserver patch

.config:

CONFIG_USB_UAS=m CONFIG_NF_CONNTRACK_ZONES=y CONFIG_NF_CONNTRACK_TIMEOUT=y CONFIG_NF_CONNTRACK_TIMESTAMP=y CONFIG_NETFILTER_XT_SET=m CONFIG_L2TP=m CONFIG_L2TP_DEBUGFS=m CONFIG_L2TP_V3=y CONFIG_L2TP_IP=m CONFIG_L2TP_ETH=m CONFIG_NET_TCPPROBE=m CONFIG_SCSI_UFSHCD=m CONFIG_SCSI_ISCI=m # CONFIG_PPPOL2TP is not set