Archive

Archive for the ‘Hacks’ Category

Boot Ubuntu from RAID

May 13th, 2016 No comments

I had lately setup RAID 5 (three 2TB disks) with i7 6700 + ASUS H170 MoBo + 32GB DDR4 RAM followed by installing Ubuntu 16.04 x64 on it.  The installation went very well with Kernel 4.4.0-21.  But, upon restart the installation was unable to move ahead as the Kernel was not able to read from the RAID device.

Basically, the issue is that the install kernel didn’t have the RAID drivers embedded within.  Upon digging for few hours, found one solution at http://ubuntuforums.org/showthread.php?t=2220063 , which didn’t work as is.  The following modification worked:

  1. Boot from Live DVD image, which would load the RAID drivers automatically allowing you to see the device correctly.
  2. Now mount the root partition. (say /mnt/root-partition)
  3. Edit /mnt/root-partition/etc/initramfs-tools/modules for adding drivers that should get embedded into Initialization RAM drive file System (initrd).
  4. Add the following list at the end of the modules file.  The list is generated using “lsmod|grep raid” during the live DVD session.
    1. dm_raid
      raid456
      async_raid6_recov
      async_memcpy
      async_pq
      async_xor
      async_tx
      id6_recov
      raid6_pq
      libcrc32c
  5. Run “mkinitramfs -r /mnt/root-partition -o /mnt/root-partition/boot/initrd.img-4.4.0-21-generic”
  6. Reboot the machine to see that RAID is being detected.
  7. Install dmraid (apt-get install dmraid), which would again fix the initrd image with necessary stuff.
  8. Now, my machine works smooth.

If nothing worked, get a small HDD to install the OS in it.  While booting the OS, you may load the drivers via /etc/rc.local and dmraid commands to activate the RAID device and mount it for use.

Installing nvidia driver in MSI GE72 6QD

February 11th, 2016 No comments
  1. Uninstall all nvidia drivers.
  2. Change GRUB_CMDLINE_LINUX_DEFAULT to:
    GRUB_CMDLINE_LINUX_DEFAULT="quiet splash i915.preliminary_hw_support=1 i915.enable_execlists=0 intel_pstate=disable i915.enable_rc6=0"
    
  3. Then run sudo update-grub2 to make the change permanent.
  4. Restart your laptop to witness the proper function of video screen, which is using the Intel GPU now.
  5. Now you can change the screen brightness using the Fn+Brightness key. Previous to this change, my screen brightness remained the same always.
  6. Install the latest nvidia driver using apt-get. In my case, I installed 361.18 and my Kernel was 4.2.0-27
  7. Restart now and see nvidia driver in action. I had checked the functionality by running nvidia-settings.

Reference: https://gist.github.com/ngangat/02c8279e49783328de3a

Update:  In 16.04 Xenial release, I get a lot of the following errors

PCIe Bus Error: severity=Corrected type=PhysicalLayer, id=00e0(Receiver ID)
device [8086:a110] error status/mask=0000001/00002000
[ 0] Receiver Error

This problem can be overcome by adding pci=noaer to GRUB_CMDLINE_LINUX_DEFAULT.

Automated Linux GUI Applications using X11::GuiTest

November 3rd, 2015 No comments

Lately, I wanted to extract from application-embedded medical lexicon to construct a database for easy use of the lexicon data.  The application was ancient (1998) and Windows 32bit.  The application does not work in Windows version beyond Windows XP 32bit.  But fortunately, the application was very runnable in Ubuntu 14.04 over Wine layer.  A snapshot of the application is shown below:

app

There were several restrictions in the application, viz.,  “Select All” does not work, direct copy-paste does not work.  But the traversal of lookup items on the left pane could be fully done using keyboard keys.  So, the process for extracting every item is:

  1. TAB + TAB to go to the left pane containing the lookup items
  2. TAB to move to the right pane.
  3. Ctrl + Shift + End would select the entire text on the right pane.
  4. Ctrl + C copies the selected text to the clipboard
  5. Using “xclip”, it is possible to get the clipboard data into the driver script.
  6. Save the recovered clipboard content to a file.
  7. Move back to the left pane by Shift + TAB.
  8. Down arrow to move to the next item in the list.
  9. Goto Step 2, until exhaustion.

The ideal toolkit for this project is the X11::GuiTest library in Perl (https://metacpan.org/pod/X11::GUITest). The same toolkit is available in software repository, which could be installed using sudo apt-get install libx11-guitest-perl.  The codeblock that I’d used in Perl for getting is done is listed below.

# Start the application 
StartApp('/path/to/app');

# wait unconditionally for 5 seconds to allow the app to come to front.
WaitSeconds(5);

# move the cursor to the scroll list
SendKeys('{TAB 2}');
sleep(2);

for (my $i = 0; $i < 20; $i++ )
{
 # move the cursor to the right pane
 SendKeys('{TAB}');

 # select all
 SendKeys('^(+({END}))');

 # perform copy
 SendKeys('^(c)');
 
 # read the data from clipboard and print that in stdout
 my $clipboard = `xclip -selection clipboard -o`;

 if ( $clipboard ne $clipboard_prev ) { print '-'x 80; print "\n$i\t$clipboard\n" };
 $clipboard_prev = $clipboard;
 
 # move back to scroll list
 SendKeys('+({TAB})');

 # goto next item in the list.
 SendKeys('{DOWN}');
 WaitSeconds(0.7);
}

 

Kubuntu 15.04 Playback Control Media Keys not working

July 24th, 2015 No comments

For some reason, while the other 2nd functions keys were working fine, the Play/Pause, Previous & Next keys stopped working in Kubuntu 15.04 since the clean installation. Volume buttons were working fine.  Upon lot of digging, finally the solution was just a workaround by using MPRIS-remote.

sudo apt-get install mpris-remote

Goto System Settings->Shortcuts->Custom Shortcuts

Edit -> New Group -> “Media Player”

Edit -> New -> Global Shortcut -> Command/URL

Enter “Media Play” for the command name.

On the right pane, Click on “Trigger” -> Set the shortcut key;

Click on “Action”-> “/usr/bin/mpris-remote pause”.

Likewise create commands “Media Next” : “/usr/bin/mpris-remote next”

and “Media Previous” : “/usr/bin/mpris-remote previous”.

Clementine is a MPRIS compatible player, which means this hack solves the problem!

 

RAID Mayhem in Linux Mint 17.1 and Kubuntu 14

March 11th, 2015 No comments

I have a Quad-core machine that has 2+2 1TB disks configured in RAID 1+0 mode (mirror + stripe).  I am having a 64Gb SSD, on which I have the operating systems installed.  Recently, I wanted to install Linux Mint 17.1 on the SSD.  The installation went fine, but after rebooting the RAID device was not visible at all.  To overcome this problem, I had to do “dmraid -ay” to activate the RAID device in my machine.

Ok, now my RAID device is visible.  Went ahead to create /home and /data partitions, followed by updating the same in /etc/fstab for auto-mounting during boot-up.

To my surprise, the reboot failed saying /home is not ready.  Checked into recovery mode by punching in the root password and found that the RAID device has gone missing now.   Ran “dmraid -ay” again to activate the device; following by running “mount -a” make /home and /data partitions come alive.  Exited the recovery mode to complete the full booting.

Did some internet search for how to activate the RAID device on booting automatically.  It was awful that there are no direct methods to do so.   Infact, some people have advised about not rebooting the machine at all..  Funny!

Later, found a workaround, where the /etc/fstab entries were made “noauto”.  And RAID activation happening in /etc/rc.local.

/etc/rc.local

dmraid -ay
mount /home
mount /data

/etc/fstab

/dev/mapper/pdc_cjcdcdeide6    /home ext4    defaults,noauto     0   2
/dev/mapper/pdc_cjcdcdeide5    /data ext4    defaults,noauto     0   2

Moral: Never install the Operating System on a RAID device, as majority of the distributions don’t recognize them by default.  Interestingly, Fedora 21 recognized RAID during installation itself, but I ended up hating it, when I found “yum update” not working out of the box.

How to make Kate remember last opened files

October 22nd, 2014 No comments

Kate allows document editing to be grouped under separate sessions.  Although there is no direct configuration setting to make Kate auto-open last opened files during startup, the Sessions features come handy.  The workaround is; given that Kate remembers last opened file paths in the File -> Open Recent menu item, you may select files one by one to continue editing.  But, this is so boring.

The procedure is the following:

  1. Open all the files that you want to be remembered by Kate for reopening during next restart of Kate.
  2. Click on Sessions->Save Session As->[give a session name here to save the session]
  3. Close your Kate application.
  4. Open it again, you would see no files reopened automatically, Don’t Panic!
  5. Click on Sessions->Quick Open Session->[session name].
  6. Aha… all the files that you had put into the session will reappear.

How fast is integer arithmetic than floating point arithmetic?

June 19th, 2013 No comments
You may already know that integer arithmetic is faster than floating point arithmetic. But how fast? I had the same question in my mind. So created the following script to test the “fastness” for each data type.

#include <vector>
#include <iostream>
#include <cstdlib>
#include “Stopwatch.hpp”

template <typename T>
void test( size_t inArraySize, size_t inCorpusSize )
{
    typedef std::vector<T> array_t;
    typedef std::vector<array_t> arrays_t;

    std::cout << “constructing the corpus…”;

    Stopwatch sw;
    sw.Start();
    arrays_t corpus( inCorpusSize + 1);
    for ( size_t i = 0; i < inCorpusSize+1; ++i )
    {
        array_t &array = corpus[i];
        array.resize( inArraySize );
        for ( size_t j = 0; j < inArraySize; ++j )
        {
            array[j] = (T)( rand() );
        }
    }

    sw.Stop();
    std::cout << sw.Elapsed() << ” secs” << std::endl;

    std::cout << “scored in … “;
    sw.Start();
    array_t scores( inCorpusSize );
    const array_t &query = corpus[0];
    for ( size_t i = 1; i < inCorpusSize+1; ++i )
    {
        const array_t &array = corpus[i];

        register T &score = scores[i1];
        score = 0;
        for ( size_t j = 0; j < inArraySize; ++j )
        {
            score += array[j]*query[j];
        }
    }

    sw.Stop();
    std::cout << sw.Elapsed() << ” secs” << std::endl;

}

int main( int argc, char **argv )
{
    if ( argc < 2 )
        return 1;

    int csize = atoi(argv[1]);
    int asize = atoi(argv[2]);

    std::cout << “testing float..” << std::endl;
    test<float>( asize, csize );

    std::cout << “testing double..” << std::endl;
    test<double>( asize, csize );

    std::cout << “testing long double..” << std::endl;
    test<long double>( asize, csize );

    std::cout << “testing int..” << std::endl;
    test<int>( asize, csize );

    std::cout << “testing unsigned int..” << std::endl;
    test<unsigned int>( asize, csize );

    std::cout << “testing short..” << std::endl;
    test<short>( asize, csize );

    std::cout << “testing unsigned short..” << std::endl;
    test<unsigned short>( asize, csize );

    std::cout << “testing char..” << std::endl;
    test<char>( asize, csize );

    std::cout << “testing unsigned char..” << std::endl;
    test<unsigned char>( asize, csize );

    std::cout << “testing long long..” << std::endl;
    test<long long>( asize, csize );

    std::cout << “testing unsigned long long..” << std::endl;
    test<unsigned long long>( asize, csize );

    return 0;
}

Syntax highlighting done by: /usr/bin/highlight -O html -t 4 -I -S cpp Main.cpp –inline-css -o /tmp/m.html

The result of running this script with vector size = 100 and corpus size = 1 million is shown below:
sudar@kiriya /opt/testing/bin $ ./tester 1000000 100
testing float..
constructing the corpus…1.447 secs
scored in … 1.731 secs

testing double..
constructing the corpus…1.543 secs
scored in … 1.836 secs

testing long double..
constructing the corpus…2.135 secs
scored in … 2.614 secs

testing int..
constructing the corpus…1.454 secs
scored in … 1.657 secs

testing unsigned int..
constructing the corpus…1.41 secs
scored in … 1.615 secs

testing short..
constructing the corpus…1.394 secs
scored in … 1.59 secs

testing unsigned short..
constructing the corpus…1.347 secs
scored in … 1.542 secs

testing char..
constructing the corpus…1.313 secs
scored in … 1.507 secs

testing unsigned char..
constructing the corpus…1.328 secs
scored in … 1.522 secs

testing long..
constructing the corpus…1.553 secs
scored in … 1.767 secs

testing unsigned long..
constructing the corpus…1.548 secs
scored in … 1.761 secs

  1. If you observe carefully, floating point calculations have taken more time than the fixed point calculations.
  2. Long Double took the longest time and it is the slowest.
  3. The reverse climax is that 64bit fixed calculation is slower than single-precision arithmetic.  An interesting read on multiplication is here.  The reason could be overflow due to the size of product register being 64 bits.
  4. The fastest is 8 bit calculation.  But unsigned 8 bit arithmetic is slower than signed.  Can’t explain why!
  5. For fixed point, unsigned calculations take less time than signed calculations, which can be understood by the additional time taken for 2’s complement arithmetic.
  6. Results can be different with different CPUs / Operating Systems, so infer accordingly.

Upgrade version of the testing application.
#include <vector>
#include <iostream>
#include <cstdlib>
#include “Stopwatch.hpp”
#include <string>
#include <map>

template <typename T>
double test( size_t inArraySize, size_t inCorpusSize )
{
     typedef std::vector<T> array_t;
     typedef std::vector<array_t> arrays_t;

     arrays_t corpus( 2 );
     for ( size_t i = 0; i < 2; ++i )
     {
          array_t &array = corpus[i];
          array.resize( inArraySize );
          for ( size_t j = 0; j < inArraySize; ++j )
          {
               array[j] = (T)( rand()%1000 );
          }
     }

     Stopwatch sw;
     sw.Start();
     const array_t &query = corpus[0];

     array_t scores( inCorpusSize );
     for ( register size_t i = 0; i < inCorpusSize; ++i )
     {
          const array_t &docu = corpus[1];
          T & score = scores[i];
          score = 0;
          for ( register size_t j = 0; j < inArraySize; ++j )
          {
               score += docu[j]*query[j];
          }
     }

     sw.Stop();
     return sw.Elapsed();
}

int main( int argc, char **argv )
{
     typedef double (*fun_t)( size_t, size_t );

     typedef std::pair < std::string, fun_t > named_fun_t;
     std::vector< named_fun_t > functions;

     functions.push_back( named_fun_t( “float”, test<float>) );
     functions.push_back( named_fun_t( “double”, test<double>) );
     functions.push_back( named_fun_t( “long double”, test<long double>) );
     functions.push_back( named_fun_t( “int”, test<int>) );
     functions.push_back( named_fun_t( “unsigned int”, test<unsigned int>) );
     functions.push_back( named_fun_t( “short”, test<short>) );
     functions.push_back( named_fun_t( “unsigned short”, test<unsigned short>) );
     functions.push_back( named_fun_t( “long long”, test<long long>) );
     functions.push_back( named_fun_t( “unsigned long long”, test<unsigned long long>) );
     functions.push_back( named_fun_t( “char”, test<char>) );
     functions.push_back( named_fun_t( “unsigned char”, test<unsigned char>) );

     if ( argc < 4 )
     {
          std::cout << argv[0] << ” <corpus_size> <array_size> <repeats>” << std::endl;
          return 1;
     }

     int csize = atoi(argv[1]);
     int asize = atoi(argv[2]);
     int iters = atoi(argv[3]);

     typedef std::map<std::string, double> timings_t;
     timings_t timings;

     for ( size_t j = 0; j < iters; ++j )
     {
          std::cout << \niteration #” << j+1 << std::endl;
          for ( size_t i = 0; i < functions.size(); ++i )
          {
               std::string name = functions[i].first;
               double timetaken = functions[i].second(asize, csize);

               timings[name] += timetaken;
               std::cout << “testing “ << name << “….” << timetaken << std::endl;
          }
     }

     std::string minstring;
     double minvalue = 99999.9;
     std::cout << \naverages” <<std::endl;
     for ( timings_t::iterator iter = timings.begin(), end = timings.end(); iter != end; ++iter )
     {
          iter->second /= (double)iters;
          std::cout << “average time for “ << iter->first << “: “ << iter->second << std::endl;

          if ( iter->second < minvalue )
          {
               minvalue = iter->second;
               minstring = iter->first;
          }
     }

     std::cout << “minimum timing is for “ << minstring << “: “ << minvalue << std::endl;
}

The result of running this script with vector size = 500 and corpus size = 1 million with 20 iterations is shown below:

iteration #1
testing float….1.473
testing double….1.469
testing long double….2.025
testing int….1.024
testing unsigned int….1.069
testing short….0.996
testing unsigned short….0.993
testing long long….1.004
testing unsigned long long….1.004
testing char….0.991
testing unsigned char….0.991

iteration #2
testing float….1.449
testing double….1.459
testing long double….2.011
testing int….1.002
testing unsigned int….1
testing short….0.985
testing unsigned short….0.99
testing long long….0.993
testing unsigned long long….0.993
testing char….0.983
testing unsigned char….0.992


..
iteration #20
testing float….1.448
testing double….1.45
testing long double….1.975
testing int….0.99
testing unsigned int….0.991
testing short….0.989
testing unsigned short….0.986
testing long long….0.997
testing unsigned long long….0.992
testing char….0.992
testing unsigned char….0.988

averages
average time for char: 0.99575
average time for double: 1.4665
average time for float: 1.4676
average time for int: 1.0046
average time for long double: 2.00115
average time for long long: 1.00585
average time for short: 0.99985
average time for unsigned char: 0.9945
average time for unsigned int: 1.0085
average time for unsigned long long: 1.0025
average time for unsigned short: 0.9963
minimum timing is for unsigned char: 0.9945

So, even in this test, char is the best choice!!

LED Emergency Lamp

June 16th, 2013 No comments

Here is the LED emergency lamp that’s being sold for few hundreds in the market.  See below for the pictures.  This lamp comes with 60+ LED lamps adding upto a power of ~4.5W.

I have attempted to reverse engineer the circuitry and drawn the circuit using ExpressSCH tool. Click on the image to see the full sized circuit.

All the components listed in the circuit are available in common electronics spares shops. 

Clicking noise from HDD??

April 22nd, 2013 No comments

Have you wondered replaced your hard drives (HDD), when there is “clicking” noise from the HDD?  When you hear that noise, it means the drive is performing a “hard-reset” internally.   I have always thought that the disk is a goner, when there is too much clicking noise from the HDD.

Lately, it was found that when the gave the HDD the 5 pin power cord, the clicking noise is gone.  In general, 4 pin power has <+12v, 0v, 0v, +5> and the 5 pin power cord has an additional 3.3v orange line.  Another discovery recently is that, the clicking noise is gone when I changed the SMPS.  When the SMPS is over loaded or so, I think it is not able to supply the current that the HDDs want, which in turn makes the HDDs to perform a hard reset internally.

So, before applying for a replacement HDD, please do try changing the power cord to a 5pin cord and when you cannot try replacing the SMPS.  Typically, when you have lot of devices in your machine, ensure you use a reasonably powerful SMPS to avoid outages.

1KWp Solar Power Plant

January 30th, 2013 22 comments

Phew! What a relief!

I have just finished installing a 1KWp solar power plant in my apartment.  It was an wonderful and tiring experience getting everything to work together, ofcourse with the help of several kind people who helped moving on with the work.  I think, my installation is the first domestic installation for a capacity >1KWp in my entire area, that is Thirumullaivoyil; may be the first in Ambattur + Avadi townships.  Whatever be it, I am very happy that I could get it up and running as per the plan.  The power plant can produce upto 4-5KWh everyday in winter+sunny and may be 5-6KWh in summer.  I have been able to attach everything in my home except for Fridge, Ovens, Water heater, A/C and washing machine.  Yeah! Mixie, Coffee Maker, Wet Grinder, Computers, Water purifier run in solar at my home!  I tried attaching Fridge to the unit, but the backup time in the nights drastically reduced.  By the way, the solar power supply is 24hours through out.  The secret is that the excess power produced in the day are backed up in huge batteries, so that the saved energy is dispensed through out the night for powering my bed room and other low power night utilities.

Things to procure for this DYI project:

  1. Solar Panels: There are several variety of panels available, but always choose polycrystalline (aka multi-crystalline) panels, as there are very efficiently within 30 degree deviation of sunlight inclination angle.  If you go for monocrystalline, they are efficient only when the sun is right on top of them, so you are forced to install a solar tracking device for an additional cost.  I had procured MNRE approved panels for Rs 46/watt including transportation and taxes from Akshayasolar.  My setup has 12V 150Wp x 6 and 100Wp x 2, put together 1100Wp of bright power.
  2. Inverter: There are several varieties of inverters available in the market. Solar Inverters are the ideal choice here, but for my specification I could not get a solar inverter within my budget.  Typically solar inverters are priced around Rs 30000 for a 1KW capacity.  So, I had decided to go to regular sine-wave inverters and convert that to a solar inverter myself.  To convert an inverter to a solar inverter, I had to get a solar charge controller and fabricate a control circuitry.  I have bought a 24V, 1.4KVA Amaron Inverter for Rs 7000, with 2 years warranty.  A typical inverter has a power-factor of 0.7 to 0.8, so I could load upto 1000W comfortably without frying the inverter.
  3. Batteries: This is the most critical part of any ESU (Energy Storage Unit), where the power is stored during surplus and released when solar incoming reduces.  On the contrary, there are EEU (Energy Export Unit), which directly converts the energy derived by the solar panels into useful power for consumption or resale back to the grid.  TANGEDCO does not buy from installations less than 1MWp, so ESU is my only option.  So storing the power, the popular choice is Exide Solar Tubular batteries.  But, I chose Sharana Batteries because their manufacturing unit is in Ambattur Industrial Estate, and their service guarantee was better than what Exide offered.  While buying batteries, one should be very careful about the capacity rating of the batteries.  By capacity, the common misnomer is the AH rating.  AH rating tells the current limits of a battery, but the capacity rating is mentioned as Cxx (eg: C10, C20, etc).  Typically, a 180AH/C20 battery is same as the 150AH/C10 battery or 100AH/C5.  In C20 rating, if the drain current should be 9A, the battery would supply energy for 20 hours.  The same battery would serve for 8.33 hours only if the drain current is 18A, but you would expect it to serve for 10 hours.  Likewise, the same battery would serve only for 2.77 hours, if the drain current was 36A where you would expect a backup time of 5 hours (see here the capacity has halfed!!).  I have purchased 2x180AH @C20 tall tubular batteries from Sharana with 2 years warranty for Rs 26000.
  4. Charge Controller: There are two types of charge controllers available namely PWM, MPPT. Solar panels, although rated as 12V, they generate voltages upto 21.6V (open-circuit).  They are optimal at around 16.4-17.4V, which is called the maximum power point.  In order to leverage the maximum power from the controller, I used MPPT charge controllers.  For 1.1KW at 24V, the Imax is 50A, but assuming 80% efficiency for panels, 40A MPPT is sufficient.  I got 40A 24V controller from Adaptek India, Adyar for Rs 13600 with 3 years warranty.
  5. Structure Fabrication: This is the messy part of the entire exercise as readymade units were not available.  Since I will be using 8 panels in total and I had to use just 100 sq.ft of space on the roof, I planned to build a two row beam structure to take the load.  The weight of a panel is roughly 10kg, to 40kgs per structure is the setting.  The design shall have a rectangular frame of size 10’x3′ made out of 1.5″ L section of steel.  The rectangular section shall be supported by two 2″x1″ C channel steel of height 1.5′, which I call the legs.  The legs shall rest on a 8″x8″ 6mm plate, which in turn shall be secured to the roof with four 10mm anchor-flush bolts.  I could get raw material and fabrication done in Mannurpet, Ambattur for Rs 8000.  The total weight of steel I had installed is about 50Kg to keep the 70Kg of panel structure load secured.

Solar Power Plant Sizing chart is here.

More to continue..