I now have four six-port Qotom systems and one four-port Qotom system in my rack. I have successfully verified that the newest of these is capable of pulling about 745Mbit/s of 1500-byte frames through the switch from six other hosts. I bet it will push data at the same rate. but I can’t get iperf to distribute it over the LAG evenly. That’s another problem for another day. I know it can handle six large streams at the same time, which is great.
I am recovering some tapes from back in the day that some of you may enjoy. Here is a log of the process so that maybe you can recover some of your own DV tapes. Seems to work well in modern Debian.
To attach to the camcorder, I used a PCI-e card that has an old firewire port and some ASIC on board. The PCI card came up and loaded the correct kernel drivers.
Here is a search link so that you can buy a similar card.
cjac@server0:~$ sudo lspci | grep 1394
b2:00.0 FireWire (IEEE 1394): VIA Technologies, Inc. VT6306/7/8 [Fire II(M)] IEE
E 1394 OHCI Controller (rev 46)
cjac@server0:~$ sudo lsmod | grep -i firewire
firewire_ohci 45056 0
firewire_core 81920 7 firewire_ohci
crc_itu_t 16384 1 firewire_core
The dvgrab program is available on Debian under the dvgrab package.
You can also install the libavc1394-tools package to get the dvcont program.
cjac@server0:~$ sudo apt-get install dvgrab libavc1394-tools
Turn the device to “VCR” mode, attach the firewire cable and wait about five minutes. Have you watered the cat today?
cjac@server0:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 10 (buster)
cjac@server0:~$ uname -r
cjac@server0:~$ sudo modinfo firewire_ohci | grep vermagic
vermagic: 5.2.0-0.bpo.3-amd64 SMP mod_unload modversions
cjac@server0:~$ dvcont status
cjac@server0:~$ dvcont rewind
cjac@server0:~$ dvcont status
cjac@server0:~$ dvcont status
# make a directory to store the raw dv tape data and the
cjac@server0:~$ mkdir -p /srv/nfs/cj.backup/dv/oscon2006
# I’ve found that each tape stores around 12 GB of raw data, so be
# sure to perform this on a partition with tens of gigs of spare
cjac@server0:~$ cd /srv/nfs/cj.backup/dv/oscon2006
cjac@server0:/srv/nfs/cj.backup/dv/oscon2006$ dvgrab –autosplit –timestamp –size 0 –rewind oscon2006-
Found AV/C device with GUID 0x0000850000e043cf
Waiting for DV…
“oscon2006-2006.07.26_12-37-44.dv”: 266.30 MiB 2327 frames timecode 00:01:17.26 date 2006.07.26 12:39:01
“oscon2006-2006.07.26_12-40-59.dv”: 816.76 MiB 7137 frames timecode 00:05:16.01 date 2006.07.26 12:44:57
“oscon2006-2006.07.26_12-45-06.dv”: 8420.56 MiB 73580 frames timecode 00:46:11.05 date 2006.07.26 13:26:01
“oscon2006-2006.07.26_13-32-08.dv”: 2961.27 MiB 25876 frames timecode 00:00:00.00 date 2020.06.10 10:46:25
During the capture, the dvcont status will be “Playing”:
cjac@server0:/srv/nfs/cj.backup/dv/oscon2006$ dvcont status
In a different window of the screen session or I guess a new gnome-terminal, put together a transcoding environment.
cjac@server0:/srv/nfs/cj.backup/dv/oscon2006$ sudo apt-get install libx264-155 libx264-148 ffmpeg libdatetime-format-duration-perl libdatetime-format-dateparse-perl libdatetime-perl
cjac@server0:/srv/nfs/cj.backup/dv/oscon2006$ wget https://raw.githubusercontent.com/cjac/dvscripts/master/transcode.pl && chmod u+x transcode.pl
# review transcode.pl, change $prefix
The script will detect partial transcodes and do the right thing generally, so don’t worry too much about running ./transcode.pl too often.
Results are being stored in various places including
A couple years ago, I bought a 12G/s SAS disk to see if I could get it to work with the RAID controller with external SFF-8088 ports which came with the system I got while I was working at The Linux Foundation. I got an enclosure to go with it because I was enthusiastic and optimistic about my ability to get things all set up. My plan was to take the 1T disks I had in the storage server I had been using before, but which had since failed and to put them into this new enclosure. I got a couple of SFF-8088 cables, and the enclosure had some SFF-8088 to 7-pin SATA break-outs come with it. I bought an additional dual SFF-8088 to SFF-8087 adapter and a couple of SFF-8087 to 4x SFF-8482 cables. I got the 8482s because I imagined that these would be required to take full advantage of the 6TB 12G/s SAS disk mentioned above.
Unfortunately, when I attached the disks to the controller, none of them worked and I gave up frustrated and disillusioned for a good long time. Last week was spring break, and I took a few days off from work to hang out with my girls. While we weren’t all goofing off together, I was puttering around with a new HBA I got to replace the apparently non-functional HP RAID controller which came with the server.
However, when I removed the HP RAID controller and put in the LSI 9201-16E HBA I got to replace it with, there was again no SCSI love. The lsscsi command showed nothing. But the lspci command showed the controller, and I started digging through documentation to figure out what might be wrong. I eventually stumbled upon a firmware flashing ISO disk image on the Broadcom site. Broadcom apparently purchased LSI not too long ago and is now responsible for managing documentation and downloads for the legacy devices, of which I am now a proud owner.
I was able to build a USB disk from this ISO using unetbootin and emacs. I had to modify the syslinux.cfg file to correct the path and case of the filenames. And for some reason, unetbootin replaced the filename of the disk image from which freedos was supposed to boot with something useless. In any case, after corrections, the entry in syslinux.cfg looks like this:
menu label 9201_16e
Unfortunately, the firmware on the 9201_16E.IMG fat filesystem was old, so I loop mounted it and wrote a more recent version of the firmware, mptsas2.rom and sas2flsh.exe to it. I’ve uploaded it to 9201_16E.IMG in case anyone might find it useful. While I’m at it, I suppose I should put the whole usb boot image up for those who might need it. See lsi-flash-usb.img.bz2. To use this disk image, decompress the .bz2 file and dd it to your USB block device, for instance /dev/sdb in my case:
$ bunzip2 /tmp/lsi-flash-usb.img.bz2 | sudo dd of=/dev/sdb
My Proliant DL370 G6 server doesn’t have a fancy new UEFI system, so I used this boot disk to get access to the SAS controller in real mode. Once I booted it and selected 9201_16e (the 21st option from the top), I had to rapidly press the up or down arrow and select the EMM386 option to load the driver. For those playing along at home, I placed the more recent LSI controller firmware and BIOS in the \CJ directory, which has the same structure as the \LSI directory from the original disk image. You should be able to
Answer the prompts, and afterwards you should have a controller with P20 firmware and BIOS written to it.
Daunting as this was, it only got me most of the way through the process. All of my SATA disks were working at this point, but the SAS disk was not. After some sleuthing, I discovered that the disk was SCSI formatted to 520 byte sectors. Linux does not accept sector sizes that are not a power of 2, and so i was unable to do anything with the disk using fdisk, kpartx, etc. Research pointed me to the sg_format utility, but I was unable to make that work. Here’s the output of the command:
$ sudo sg_format --format --size 512 -vv /dev/sg4 open /dev/sg4 with flags=0x802 inquiry cdb: 12 00 00 00 24 00 SEAGATE DKS2F-H6R0SS 7FA6 peripheral_type: disk [0x0] PROTECT=0 inquiry cdb: 12 01 00 00 24 00 inquiry cdb: 12 01 80 01 00 00 Unit serial number: Z4D0M2BF0000W515S4WH inquiry cdb: 12 01 83 01 00 00 LU name: 5000c50062ba7973 mode sense (10) cdb: 5a 00 01 00 00 00 00 00 fc 00 mode sense (10): pass-through requested 252 bytes (data-in) but got 28 bytes Mode Sense (block descriptor) data, prior to changes: Mode sense number of blocks maxed out, set longlba mode sense (10) cdb: 5a 10 01 00 00 00 00 00 fc 00 mode sense (10): pass-through requested 252 bytes (data-in) but got 36 bytes Mode Sense (block descriptor) data, prior to changes: <<< longlba flag set (64 bit lba) >>> Number of blocks=11473076960 [0x2abd942e0] Block size=520 [0x208] mode select (10) cdb: 55 11 00 00 00 00 00 00 22 00 mode select (10) parameter list 00 00 00 00 01 00 00 10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 04 00 01 0a 0c 14 ff 00 00 00 05 00 mode select (10): Descriptor format, current; Sense key: Illegal Request Additional sense: Parameter list length error Descriptor type: Sense key specific: Field pointer: Error in Command: byte 7 bit 7 Descriptor type: Field replaceable unit code: 0x5 Descriptor type: Vendor specific [0x80] 00 00 00 00 00 00 00 00 00 00 00 00 00 00 Raw sense data (in hex): 72 05 1a 00 00 00 00 1c 02 06 00 00 cf 00 07 00 03 02 00 05 80 0e 00 00 00 00 00 00 00 00 00 00 00 00 00 00 MODE SELECT command: Illegal request sense key, apart from Invalid opcode
I was about to give up, but for some reason did not. I tried to find firmware and read any documentation about my disk, a Seagate ST6000NM0024. Eventually I found a github repo called ToolBin. In this repo there is a program named SeaChest_Format_121_1183_64 which seems to use a proprietary SCSI format command to tell the drive to format itself using 512-byte sectors. I can’t say for certain yet that it works, but it did not immediately fail as sg_format did. The command I used (and this will vary for you, depending on what sg_scan -i tells you is your scsi generic device) is:
./SeaChest_Format_121_1183_64 -d /dev/sg5 --fastFormat 1 --formatUnit 512 --confirm I-understand-this-command-will-erase-all-data-on-the-drive
Assuming this command results in success after the 12 hour run, I will purchase another 7 of these drives and fill out my disk array!
Sun Apr 14 13:39:48 PDT 2019 Format Unit Progress = 3.61%
Thanks to the #linux-raid channel for helping me through this!
Mon Apr 15 08:16:00 PDT 2019 Format Unit Progress = 99.97% Mon Apr 15 08:17:00 PDT 2019 Format Unit Progress = 100.00% [70874.677389] sd 3:0:2:0: device_block, handle(0x0012) [70876.176199] sd 3:0:2:0: device_unblock and setting to running, handle(0x0012) [70876.249541] mpt2sas_cm0: removing handle(0x0012), sas_addr(0x5000c50062ba7971) [70876.253344] mpt2sas_cm0: removing : enclosure logical id(0x500062b20059ecc0), slot(1) [70886.680412] scsi 3:0:3:0: Direct-Access SEAGATE DKS2F-H6R0SS 7FA6 PQ: 0 ANSI: 5 [70886.684166] scsi 3:0:3:0: SSP: handle(0x0012), sas_addr(0x5000c50062ba7971), phy(2), device_name(0x5000c50062ba7970) [70886.688833] scsi 3:0:3:0: SSP: enclosure_logical_id(0x500062b20059ecc0), slot(1) [70886.750049] sd 3:0:3:0: Attached scsi generic sg5 type 0 [70886.750143] sd 3:0:3:0: [sdd] Spinning up disk... [70887.765498] . [70888.789522] . [70889.813591] . [70890.837718] . [70891.861715] . [70892.885876] . [70893.909889] . [70894.933912] . [70895.957989] . [70896.982028] . [70898.006133] . [70899.030215] . [70900.054269] . [70901.078340] . [70902.102359] . [70903.478491] . [70904.502557] . [70906.823608] ready [70906.826165] sd 3:0:3:0: [sdd] 11721045168 512-byte logical blocks: (6.00 TB/5.46 TiB) [70906.829779] sd 3:0:3:0: [sdd] 4096-byte physical blocks [70906.834708] sd 3:0:3:0: [sdd] Write Protect is off [70906.837364] sd 3:0:3:0: [sdd] Mode Sense: 13 00 10 08 [70906.838142] sd 3:0:3:0: [sdd] Write cache: disabled, read cache: enabled, supports DPO and FUA [70906.867106] sd 3:0:3:0: [sdd] Attached SCSI disk $ sudo ./SeaChest_Info_141_1183_64 -d /dev/sg5 -i ========================================================================================== SeaChest_Info - Seagate drive utilities - NVMe Enabled Copyright (c) 2014-2018 Seagate Technology LLC and/or its Affiliates, All Rights Reserved SeaChest_Info Version: 1.4.1-1_18_3 X86_64 Build Date: Oct 18 2018 Today: Mon Apr 15 14:52:33 2019 ========================================================================================== /dev/sg5 - DKS2F-H6R0SS - Z4D0M2BF0000W515S4WH - SCSI Vendor ID: SEAGATE Model Number: DKS2F-H6R0SS Serial Number: Z4D0M2BF0000W515S4WH Firmware Revision: 7FA6 World Wide Name: 5000C50062BA7973 Copyright: Copyright (c) 2014 Seagate All rights reserved Drive Capacity (TB/TiB): 6.00/5.46 Temperature Data: Current Temperature (C): 45 Highest Temperature (C): Not Reported Lowest Temperature (C): Not Reported Power On Time: 21 days 11 hours 8 minutes Power On Hours: 515.13 MaxLBA: 11721045167 Native MaxLBA: Not Reported Logical Sector Size (B): 512 Physical Sector Size (B): 4096 Sector Alignment: 0 Rotation Rate (RPM): 7200 Form Factor (inch): 3.5 Last DST information: DST has never been run Long Drive Self Test Time: 10 hours 2 minutes Interface speed: Port 0 (Current Port) Max Speed (GB/s): 12.0 Negotiated Speed (Gb/s): 6.0 Port 1 Max Speed (GB/s): 12.0 Negotiated Speed (Gb/s): Not Reported Annualized Workload Rate (TB/yr): 0.00 Total Bytes Read (MB): 6.39 Total Bytes Written (MB): 14.99 Encryption Support: Not Supported Cache Size (MiB): Not Reported Read Look-Ahead: Not Supported Write Cache: Disabled SMART Status: Good ATA Security Information: Not Supported Firmware Download Support: Full, Segmented, Deferred Specifications Supported: SPC-3 SAM-5 SAS-3 SPL-3 SPC-4 SBC-3 Features Supported: Application Client Logging Self Test Automatic Write Reassignment Automatic Read Reassignment EPC Informational Exceptions [Mode 0] Translate Address Format Unit Sanitize
cjac@server0:~/Downloads/geth-linux-amd64-1.8.14-316fc7ec$ df -h ~/.ethereum/ Filesystem Size Used Avail Use% Mounted on /dev/mapper/vg00-ethereum 148G 130G 11G 93% /home/cjac/.ethereum
this was from before I ran
./geth --syncmode "fast" --cache=1024
Hello folks! My team spent hours and hours beating our head against a Sonar deployment problem on Ubuntu Trusty (14.04 LTS). I thought I might share our findings so that you won’t have to!
As you probably know, Trusty only makes Java Development Kit 1.7 available on the stock installation. The current stable version of the Java is 1.8. The way we install this is to use the OpenJDK PPA, generously uploaded by our dear friend Matthias Klose.
To make things even more exciting, a modern Maven is not available on this platform. And so we use the stock Maven 3.3.9 tarball distribution. This tarball distribution does not integrate well with Debian, and so, when we tell the system using
sudo update-java-alternatives -s /usr/lib/jvm/java-1.8.0-openjdk-amd64 that we wish to use Java 1.8 as our default system JDK, it does not get the message.
The only way to reliably let Maven know which java you wish to use is to set the JAVA_HOME environment variable. In order to do this within the Jenkins environment, one must select the JDK one wishes to use:
To make things worse, this option is not, as one might expect, available for editing in a stock Jenkins 2.x installation. In Jenkins 1.x, one would be able to specify which java one wished to use just by specifying “openjdk8” in a field. With Jenkins 2.x, the field does not exist unless a configuration option in an unrelated form is set.
So! One should first select Manage Jenkins -> Global Tool Configuration:
Once this form is open, look for the “JDK installations…” button:
Click it very thoroughly just once.
You’ll be presented with a form into which you may enter details about the various JDKs your build executors may have access to. You’ll refer to them in your job configuration by the value of their “Name” field, and when executing the build, Jenkins will set JAVA_HOME to the value of the (you guessed it) JAVA_HOME field:
Once these entries are made, they can be selected in two place.
1) on the ZMQ Event Publisher:
2) in the post-build actions under SonarQube analysis with Maven (advanced)
And that’s how it’s done!
Here’s the details from my colleague, Thanh:
I’ll write this later. Keywords and notes inline
Developers and users working together.
Fast Dev/Test cycles
Sessions start at 10:30
Minimal install needs to be smaller
Most functionalities must be implemented in modules
Upgrade path complexity
Out of band control channel.
OPNFV customers don’t run stock releases, make customisations and plugins easier.
Distribution vendors should not be expected to perform package maintenance. Get the distribution ready for fast pathing in to backports, security, testing and unstable.
Looks like I was missing qemu-kvm.
$ sudo apt-get install qemu-kvm qemu-system
I put some packages together this weekend. It’s been a while since I’ve debuilt anything officially.
The plan is to build a binding to the libgnutls.so.30 API. The certtool CSR (REQ) generation interface does not allow me to create a CRL with “not critical” attributes set on purposes. Maybe if I do it a bit closer to the metal it will be easier…
“Why?”, you might ask, would one want to run something other than the most recent version of Maven on the very newest and fangledest breed of the linux distribution we have all loved for so long.
“Because!”, I might answer, I’m trying to get the nexus-apt-plugin working on nexus.fd.io, and the version of nexus we’re running there explained to me in quite uncertain terms that it would talk to no other version of maven than 3.0.4 or something else that is not packaged for debian.
So I grabbed the source for version 3.0.4 from wheezy and patched it up to work with stretch:
$ cd /usr/src/deb $ dget http.debian.net/debian/pool/main/m/maven/maven_3.0.4-3+deb7u1.dsc $ cd maven-3.0.4 $ perl -i -pe 's/(libmodello-maven-plugin)1.4(-java)/$1$2/' debian/control $ quilt pop -a $ quild push 1 $ perl -i -pe 's/-1.4.x\.jar/.jar/' build.xml $ perl -i -pe 's/google-collections/guava/' build.xml $ perl -i -pe 's/\s+$//' build.xml $ quilt refresh $ quilt pop $ quilt push -a $ debuild -uc -us $ sudo apt-get remove maven libmaven3-core-java $ sudo dpkg -i ../maven_3.0.4-3+deb7u1_all.deb
And now I can build the silly nexus-apt-plugin…
$ mkdir -p /usr/src/git/github $ git clone firstname.lastname@example.org:LLC-Technologies-Collier/nexus-apt-plugin.git /usr/src/git/github/nexus-apt-plugin $ cd /usr/src/git/github/nexus-apt-plugin $ mvn compile && mvn -q test