How to set CPU affinity for a process in FreeBSD

September 2, 2010 at 4:29 pm | FREEBSD, HOW-TOS | 1 comment

beastieProcessor affinity means, on a multi-CPU machine, the process(es)run only on dedicated set of CPUs. In other words processes are bound to isolated (subset) of the CPUs. This feature can be usedduring performance benchmarking, and also while deploying an application.

To get the CPU model and number of active CPUs try the following command:

$ sysctl hw.model hw.ncpu

The cpuset command

The cpuset command can be used to assign processor sets to processes, run commands constrained to a given set or list of processors, and query information about processor binding, sets, and available processors in the system.

cpuset requires a target to modify or query. The target may be specified as a command, process id, thread id, a cpuset id, an irq or a jail id. Using -g the target’s set id or mask may be queried. Using -l or -s the target’s CPU mask or set id may be set. If no target is specified, cpuset operates on itself. Not all combinations of operations and targets are supported. For example, you may not set the id of an existing set or query and launch a command at the same time.

There are two sets applicable to each process and one private mask per thread. Every process in the system belongs to a cpuset. By default processes are started in set 1. The mask or id may be queried using -c. Each thread also has a private mask of CPUs it is allowed to run on that must be a subset of the assigned set. And finally, there is a root set, numbered 0, that is immutable. This last set is the list of all possible CPUs in the system and is queried using -r.

When running a command it may join a set specified with -s otherwise a new set is created. In addition, a mask for the command may be specified using -l. When used in conjunction with -c the mask modifies the sup- plied or created set rather than the private mask for the thread.

The options are as follows:

-c           The requested operation should reference the cpuset avail-
             able via the target specifier.

-g           Causes cpuset to print either a list of valid CPUs or, using
             -i, the id of the target.

-i           When used with the -g option print the id rather than the
             valid mask of the target.

-j jailid    Specifies a jail id as the target of the operation.

-l cpu-list  Specifies a list of CPUs to apply to a target.  Specifica-
             tion may include numbers seperated by '-' for ranges and
             commas separating individual numbers.

-p pid       Specifies a pid as the target of the operation.

-s setid     Specifies a set id as the target of the operation.

-r           The requested operation should reference the root set avail-
             able via the target specifier.

-t tid       Specifies a thread id as the target of the operation.

-x irq       Specifies an irq as the target of the operation.

Examples

Create a new group with CPUs 0-4 inclusive and run /bin/sh on it:

cpuset -c -l 0-4 /bin/sh

Query the mask of CPUs the is allowed to run on:

cpuset -g -p

Restrict /bin/sh to run on CPUs 0 and 2 while its group is still allowed
to run on CPUs 0-4:

cpuset -l 0,2 -p

Modify the cpuset /bin/sh belongs to restricting it to CPUs 0 and 2:

cpuset -l 0,2 -c -p

Modify the cpuset all threads are in by default to contain only the first
4 CPUs, leaving the rest idle:

cpuset -l 0-3 -s 1

Print the id of the cpuset /bin/sh is in:

cpuset -g -i -p

Move the pid into the specified cpuset setid so it may be managed with
other pids in that set:

cpuset -s  -p

Source: http://www.freebsd.org/cgi/man.cgi?query=cpuset

PDF Manipulations And Conversions From Linux Command Prompt

July 14, 2010 at 9:08 pm | HOW-TOS | 2 comments

pdf

If PDF is electronic paper, then pdftk is an electronic staple-remover, hole-punch, binder, secret-decoder-ring, and X-Ray-glasses. Pdftk is a simple tool for doing everyday things with PDF documents. Pdftk allows you to manipulate PDF easily and freely. It does not require Acrobat, and it runs on Linux, Windows, Mac OS X, FreeBSD and Solaris.

In Debian/Ubuntu you can install pdftk via apt:

$ sudo aptitude install pdftk

Examples:
Merge Two or More PDFs into a New Document

$ pdftk 1.pdf 2.pdf 3.pdf cat output 123.pdf

or (Using Handles):

$ pdftk A=1.pdf B=2.pdf cat A B output 12.pdf

or (Using Wildcards):

$ pdftk *.pdf cat output combined.pdf

Split Select Pages from Multiple PDFs into a New Document

$ pdftk A=one.pdf B=two.pdf cat A1-7 B1-5 A8 output combined.pdf

Rotate the First Page of a PDF to 90 Degrees Clockwise

$ pdftk in.pdf cat 1E 2-end output out.pdf

Rotate an Entire PDF Document’s Pages to 180 Degrees

$ pdftk in.pdf cat 1-endS output out.pdf

Encrypt a PDF using 128-Bit Strength (the Default) and Withhold All Permissions (the Default)

$ pdftk mydoc.pdf output mydoc.128.pdf owner_pw foopass

Same as Above, Except a Password is Required to Open the PDF

$ pdftk mydoc.pdf output mydoc.128.pdf owner_pw foo user_pw baz

Same as Above, Except Printing is Allowed (after the PDF is Open)

$ pdftk mydoc.pdf output mydoc.128.pdf owner_pw foo user_pw baz allow printing

Decrypt a PDF

$ pdftk secured.pdf input_pw foopass output unsecured.pdf

Join Two Files, One of Which is Encrypted (the Output is Not Encrypted)

$ pdftk A=secured.pdf mydoc.pdf input_pw A=foopass cat output combined.pdf

Uncompress PDF Page Streams for Editing the PDF Code in a Text Editor

$ pdftk mydoc.pdf output mydoc.clear.pdf uncompress

Repair a PDF’s Corrupted XREF Table and Stream Lengths (If Possible)

$ pdftk broken.pdf output fixed.pdf

Burst a Single PDF Document into Single Pages and Report its Data to doc_data.txt

$ pdftk mydoc.pdf burst

Report on PDF Document Metadata, Bookmarks and Page Labels

$ pdftk mydoc.pdf dump_data output report.txt

Poppler is a PDF rendering library based on the xpdf-3.0 code base. The poppler-utils package contains pdftops (PDF to PostScript converter), pdfinfo (PDF document information extractor), pdfimages (PDF image extractor), pdftohtml (PDF to HTML converter), pdftotext (PDF to text converter), and pdffonts (PDF font analyzer).

Debian/Ubuntu users can install pdftk via apt:

$ sudo aptitude install poppler-utils


Convert PDF to TEXT

Pdftotext converts Portable Document Format (PDF) files to plain text.

$ pdftotext example.pdf example.txt

If textfile is not specified, pdftotext converts file.pdf to file.txt.  If text-file is ´-’, the text is sent to stdout.

To convert  page  from 3 to 7 (including 3 and 7) use:

$ pdftotext -f 3 -l 7 example.pdf example.txt

To extract only 3rd page

$ pdftotext -f 3 -l 3 example.pdf example.txt
$ pdftotext -layout example.pdf example.txt

Maintain the original physical layout of the text and output the text in reading order.

Set the  -nopgbrk option if you don’t want insert page breaks.

Uset -opw (owner password) or -upw (user password) options if the PDF file is password protected.


Extract Images From PDF

Pdfimages saves images from a Portable Document Format (PDF) file as Portable Pixmap (PPM), Portable Bitmap (PBM), or JPEG files.

Pdfimages reads the PDF file, scans one or more pages, and writes one PPM, PBM, or JPEG file for each image, image-root-nnn.xxx, where nnn is the image number and xxx is the image type (.ppm, .pbm, .jpg).

Pdfimages extracts the raw image data from the PDF file, without performing any additional transforms. Any rotation, clipping, color inversion, etc. done by the PDF content stream is ignored.

$ pfdimages example.pdf exampleimage

The above command will extract all images from example.pdf. The images will be saved in PPM format.

Use -j option to save images as JPG format

$ pfdimages -j example.pdf exampleimage

Use the -f and -l options to specify the startpage and lastpage to scan. To scan pages 3 to 7 (including 3 and 7) use:

$ pfdimages -f 3 -l 7 example.pdf exampleimage

To scan only one specific page use:

$ pfdimages -f 3 -l 3 example.pdf exampleimage

If the PDF file is password protected use -opw or -upw option:

-opw Owner password
-upw User password


Convert PDF to HTML

pdftohtml is a program that converts pdf documents into html. It generates its output in the current working directory.

Usage:

$ pdftohtml file.pdf file.html

If you want to see graphics, you’ll need to use the -c (as in “complex”) option:

$ pdftohtml -c file.pdf file.html


Convert PDF to Image

First you need to have ImageMagick installed in your machine.
To install ImageMagick in Debian/Ubuntu run the following command:

$ sudo aptitude install imagemagick

To convert pdf file to image use the ‘convert‘ command:

$ convert doc.pdf doc.jpeg

convert to tiff

$ convert doc.pdf doc.tiff

for more information look here.

Recording and copying system configuration in Debian/Ubuntu

July 9, 2010 at 7:03 pm | DEBIAN/UBUNTU, HOW-TOS | 2 comments

debianYou can make a local copy of the package and debconf selection states using dpkg and debconf-get-selections command. The debconf-get-selections command output the content of current debconf database.

You will require the “debconf-utils” packages to do that.

Install debconf-utils:

$ sudo aptitude install debconf-utils
$ sudo dpkg --get-selections '*' > selection.dpkg
$ sudo debconf-get-selections    > selection.debconf

Here, “*” makes “selection.dpkg” to include package entries for “purge” too.

You can transfer these 2 files to another computer, and install there with the following.

$ sudo dselect update
$ sudo debconf-set-selections < myselection.debconf
$ sudo dpkg --set-selections  < myselection.dpkg
$ sudo apt-get -u dselect-upgrade    # or dselect install

If you are thinking about managing many servers in a cluster with practically the same configuration, you should consider to use specialized package such as fai to manage the whole system.

Next Page »