Archive

Archive for October, 2010

How To Expand Usable Storage Space In Ubuntu

October 31st, 2010 No comments

ubuntu

1. Using LVM

For partitions created on Logical Volume Manager (LVM) (Linux feature) at install time, they can be resized easily by concatenating extents onto them or truncating extents from them over multiple storage devices without major system reconfiguration.

Caution: Deployment of the current LVM system may degrade guarantee against filesystem corruption offered by journaled filesystems such as ext3fs unless their system performance is sacrificed by disabling write cache of hard disk.

Run a df from terminal.

$ df
Filesystem	1K-blocks	Used	Available	Use%	Mounted on
/dev/mapper/VolGroup00-LogVol00	7935392	6773500	752292	91%	/
/dev/sda5	497829	20904	451223	5%	/boot
tmpfs	1037084	0	1037084	0%	/dev/shm
/dev/mapper/VolGroup00-LogVol01	70877776	14988144	51045372	23%	/home

We have two partitions here, / partition is about 8 Gb and the /home partition is about 71 Gb. What we are trying to do is to expand the / partition to 10 Gb by taking free space from /home.

For /home you do:

# sudo umount /home
# sudo e2fsck -f /dev/VolGroup00/LogVol01
# resize2fs /dev/VolGroup00/LogVol01 69G
# lvreduce -L-2G /dev/VolGroup00/LogVol01
# mount /home

For / you do:

# lvextend -L+2G /dev/VolGroup00/LogVol00
# resize2fs /dev/VolGroup00/LogVol00

e2fsck and resize2fs belong to package e2fsprogs.

After resizing you will get

$ df
Filesystem	1K-blocks	Used	Available	Use%	Mounted on
/dev/mapper/VolGroup00-LogVol00	9299624	6779304	2043564	77%	/
/dev/sda5	497829	20904	451223	5%	/boot
tmpfs	1037084	0	1037084	0%	/dev/shm
/dev/mapper/VolGroup00-LogVol01	68877776	14999888	51033628	23%	/home

Read the lvm-howto for detailed infotmation.

2. Mounting another partition

If you have an empty partition (e.g., “/dev/sdx”), you can format it with mkfs.ext3(1) and mount(8) it to a directory where you need more space. (You need to copy original data contents.)

$ sudo mv work-dir old-dir
$ sudo mkfs.ext3 /dev/sdx
$ sudo mount -t ext3 /dev/sdx work-dir
$ sudo cp -a old-dir/* work-dir
$ sudo rm -rf old-dir

3. Using symlink

This might be the easiest way. If you have an empty directory (e.g., “/path/to/emp-dir”) in another partition with usable space, you can create a symlink to the directory with ln(8).

$ sudo mv work-dir old-dir
$ sudo mkdir -p /path/to/emp-dir
$ sudo ln -sf /path/to/emp-dir work-dir
$ sudo cp -a old-dir/* work-dir
$ sudo rm -rf old-dir

4. Using aufs

If you have usable space in another partition (e.g., “/path/to/”), you can create a directory in it and stack that on to a directory where you need space with aufs. With aufs you can unite several directories into a single virtual filesystem.

$ sudo mv work-dir old-dir
$ sudo mkdir work-dir
$ sudo mkdir -p /path/to/emp-dir
$ sudo mount -t aufs -o br:/path/to/emp-dir:old-dir none work-dir

FreeBSD net.inet.ip Sysctls Explained

October 22nd, 2010 No comments

beastie

net.ip.portrange.*

The net.inet.ip.portrange.* sysctl variables control the port number ranges automatically bound to TCP and UDP sockets.

There are three ranges: a low range, a default range, and a high range. Most network programs use the default range which is controlled by the net.inet.ip.portrange.first and net.inet.ip.portrange.last, which default to 1024 and 5000, respectively. Bound port ranges are used for outgoing connections, and it is possible to run the system out of ports under certain circumstances. This most commonly occurs when you are running a heavily loaded web proxy. The port range is not an issue when running servers which handle mainly incoming connections, such as a normal web server, or has a limited number of outgoing connections, such as a mail relay. For situations where you may run yourself out of ports, it is recommended to increase net.inet.ip.portrange.last modestly. A value of 10000, 20000 or 30000 may be reasonable. You should also consider firewall effects when changing the port range. Some firewalls may block large ranges of ports (usually low-numbered ports) and expect systems to use higher ranges of ports for outgoing connections — for this reason it is not recommended that net.inet.ip.portrange.first be lowered.

net.inet.ip.portrange.first, net.inet.ip.portrange.last

Use the default range of values, normally net.inet.ip.portrange.hifirst through net.inet.ip.portrange.hilast. This is adjustable.

net.inet.ip.portrange.hifirst, net.inet.ip.portrange.hilast

Use a high range of values

net.inet.ip.portrange.lowfirst, net.inet.ip.portrange.lowlast

Use a low range of ports, which are normally restricted to privileged processes on UNIX systems.

net.inet.ip.portrange.reservedlow, net.inet.ip.portrange.reservedhigh

The range of privileged ports which only may be opened by root-owned processes may be modified by the net.inet.ip.portrange.reservedlow and net.inet.ip.portrange.reservedhigh sysctl settings. The values default to the traditional range, 0 through IPPORT_RESERVED – 1 (0 through 1023), respectively. Note that these settings do not affect and are not accounted for in the use or calculation of the other net.inet.ip.portrange values above. Changing these values departs from UNIX tradition and has security consequences that the administrator should carefully evaluate before modifying these settings.

net.inet.ip.portrange.randomized

Enable random port allocation. Ports are allocated at random within the specified port range in order to increase the difficulty of random spoofing attacks. In scenarios such as benchmarking, this behavior may be undesirable. In these cases, net.inet.ip.portrange.randomized can be used to toggle randomization off.

type: boolean, default: on

net.inet.ip.portrange.randomtime, net.inet.ip.portrange.randomcps

If more than net.inet.ip.portrange.randomcps ports have been allocated in the last second, then return to sequential port allocation. Return to random allocation only once the current port allocation rate drops below net.inet.ip.portrange.randomcps for at least net.inet.ip.portrange.randomtime seconds. The default values for net.inet.ip.portrange.randomcps and net.inet.ip.portrange.randomtime are 10 port allocations per second and 45 seconds correspondingly.

net.inet.ip.forwarding

Enable IP forwarding between interfaces

IP forwarding is the process of forwarding internet packets from one network to another. By default the FreeBSD system will not forward IP packets between various network interfaces. In other words, routing functions (also known as gateway functions) are disabled.

type: boolean, default: off

net.inet.ip.fastforwarding

When fast IP forwarding is enabled, IP packets are forwarded directly to the appropriate network interface with direct processing to completion, which greatly improves the throughput. All packets for local IP addresses, non-unicast, or with IP options are handled by the normal IP input processing path. All features of the normal (slow) IP forwarding path are supported including firewall (through pfil(9) hooks) checking, except ipsec(4) tunnel brokering. The IP fastforwarding path does not generate ICMP redirect or source quench messages. Compared to normal IP forwarding this can give a speedup of 40 to 60% in packet forwarding performance.

type: boolean, default: off

net.inet.ip.redirect

Enable sending IP redirects Allow (1) or disallow (0) send ICMP redirections when forwarding. This option is ignored unless the host is routing IP packets. Normally, this option should be enabled on all systems.

type: boolean, dafault: 1

net.inet.ip.ttl

The maximum time-to-live (hop count) value for an IP packet sourced by the system. This value applies to normal transport protocols, not to ICMP.

type: integer, dafault: 64

Spoofed packet attacks may also be used to overload the kernel route cache. Refer to the net.inet.ip.rtexpire, rtminexpire, and rtmaxcache sysctl parameters. A spoofed packet attack that uses a random source IP will cause the kernel to generate a temporary cached route in the route table, viewable with netstat -rna | fgrep W3.

net.inet.ip.rtexpire

Default expiration time on dynamically learned routes. Lifetime in seconds of protocol-cloned IP routes after the last reference drops.

type: integer, default: 1 hour

net.inet.ip.rtminexpire

Minimum value of ip.rtexpire. Minimum time to attempt to hold onto dynamically learned routes. This value has no effect on user modifications, but restricts the dynamic adaptation.

type: integer, default: 10 seconds.

net.inet.ip.rtminexpire

Trigger level of cached, unreferenced, protocol-cloned routes which initiates dynamic adaptation.

type: integer, default: 128

net.inet.ip.sourceroute

Enable forwarding source routed IP packets

Source Routing is a technique whereby the sender of a packet can specify the route that a packet should take through the network. Remember that as a packet travels through the network, each router will examine the “destination IP address” and choose the next hop to forward the packet to. In source routing, the “source” (i.e. the sender) makes some or all of these decisions

In strict source routing, the sender specifies the exact route the packet must take. This is virtually never used.

The more common form is loose source record route (LSRR), in which the sender gives one or more hops that the packet must go through.

type: boolean, default: false

net.inet.ip.accept_sourceroute

Enable accepting source routed IP packets

type: boolean, default: false

net.inet.ip.intr_queue_maxlen

Maximum size of the IP input queue

type: integer, default: 50

net.inet.ip.intr_queue_drops

Number of packets dropped from the IP input queue

type: integer, default: 0

net.inet.ip.maxfragpackets

Maximum number of fragmented packets the host will accept and hold in the reassembling queue simultaneously. 0 means that the host will not accept any fragmented packets. -1 means that the host will accept as many fragmented packets as it receives.

type: integer

net.inet.ip.maxfragsperpacket

Maximum number of fragments the host will accept and hold in the reassembling queue for a packet. 0 means that the host will not accept any fragmented packets.

type: integer, default: 16

net.inet.ip.fragpackets

Crrent number of IPv4 fragment reassembly queue entries.

type: integer

net.inet.ip.check_interface: Verify packet arrives on correct interface

net.inet.ip.random_id

Control IP IDs generation behaviour. The IP_ID is used for packet reassembly and needs to be unique within a certain time frame specific to a certain host. Normally the IP_ID is assigned sequentially to each IP packet leaving the host. This makes it possible to gather for example the number of hosts behind a NAT device (track different sequences of IP_ID’s). Enabling random IP_ID’s assigns a random IP_ID to each packet rendering this kind of “attack” ineffective.

Default is 0 (sequential IP IDs). IPv6 flow IDs and fragment IDs are always random.

type: boolean, default: false

net.inet.ip.sendsourcequench

Enable the transmission of source quench packets The ICMP Source quench message is a request to decrease the traffic rate of data messages sent to an internet destination. A host MAY send a Source Quench message if it is approaching, or has reached, the point at which it is forced to discard incoming datagrams due to a shortage of reassembly buffers or other resources.

type: integer, default: 0

net.inet.ip.process_options

Control IP options processing([LS]SRR, RR, TS). By setting this variable to 0, all IP options in the incoming packets will be ignored, and the packets will be passed unmodified. By setting to 1, IP options in the incoming packets will be processed accordingly. By setting to 2, an ICMP “prohibited by filter” message will be sent back in response to incoming packets with IP options. Default is 1. This sysctl(8) variable affects packets destined for a local host as well as packets forwarded to some other host.

IP Options do not have any practical use today. The only useful application is RR (Record Route) where it remembers the last 8 hops the packet traversed through. That allows you to check parts of the path back to you. IP options processing is rather expensive because the packet header has to be modified and expanded. In addition the only other use is to circumvent or trick firewalls thus it is normally blocked there.

type: integer, default: 1

net.inet.ip.keepfaith

Enable packet capture for FAITH IPv4->IPv6 translater daemon.

type: boolean, default: off

net.inet.ip.gifttl

The maximum time-to-live (hop count) value for an IP packet for gif(4) tunnel.

type: integer, default: 30

net.inet.ip.same_prefix_carp_only

Refuse to create same prefixes on different interfaces except carp interfaces.

type: boolean, default: off

net.inet.ip.subnets_are_local

Treat all subnets as directly connected

type: integer, default: 0

Categories: FREEBSD Tags: ,

FFmpeg Tricks You Should Know About

October 11th, 2010 4 comments

ffmpeg

FFmpeg is a complete, cross-platform solution to record, convert and stream audio and video. It includes libavcodec – the leading audio/video codec library. FFmpeg is free software and is licensed under the LGPL or GPL depending on your choice of configuration options.

FFmpeg supports most of the popular formats, we don’t need to worry a lot about that. Formats supported by FFmpeg include MPEG, MPEG-4 (Divx), ASF, AVI, Real Audio/Video and Quicktime. To see a list of all the codecs/formats supported by FFmpeg, run the following command:

ffmpeg -formats

1. X11 grabbing

FFmpeg can grab the X11 display.

ffmpeg -f x11grab -r 24 -s cif -i :0.0 /tmp/out.mpg

0.0 is display.screen number of your X11 server, same as the DISPLAY environment variable.

ffmpeg -f x11grab -r 24 -s cif -i :0.0+10,20 /tmp/out.mpg

0.0 is display.screen number of your X11 server, same as the DISPLAY environment variable. 10 is the x-offset and 20 the y-offset for the grabbing.

ffmpeg -f x11grab -r 25 -s 800x600 -i :0.0 /tmp/outputFile.mpg

2. Convert Pictures To Movie

First, rename your pictures to follow a numerical sequence. For example, img1.jpg, img2.jpg, img3.jpg,… Then you may run:

ffmpeg -f image2 -i img%d.jpg /tmp/a.mpg

Notice that `%d’ is replaced by the image number.

`img%03d.jpg' means the sequence `img001.jpg', `img002.jpg', etc…

If you have large number of pictures to rename, you can use the following command to ease the burden. The command, using the bourne shell syntax, symbolically links all files in the current directory that match *jpg to the `/tmp' directory in the sequence of `img001.jpg', `img002.jpg' and so on.

 x=1; for i in *jpg; do counter=$(printf %03d $x); ln "$i" /tmp/img"$counter".jpg; x=$(($x+1)); done

If you want to sequence them by oldest modified first, substitute $(ls -r -t *jpg) in place of *jpg.

Then run:

  ffmpeg -f image2 -i /tmp/img%03d.jpg /tmp/a.mpg

The same logic is used for any image format that ffmpeg reads.

3. Video Conversions

Quick and dirty convert to flv

ffmpeg -i inputfile.mp4 outputfile.flv

This converts any media ffmpeg handles to flash. It would actually convert anything to anything, it’s based on the file extension. It doesn’t do ANY quality control, sizing, etc, it just does what it thinks is best.

Convert .flv to .3gp

ffmpeg -i file.flv -r 15 -b 128k -s qcif -acodec amr_nb -ar 8000 -ac 1 -ab 13 -f 3gp -y out.3gp

Download YouTube videos as .flv and convert them to .3gp for your mobile phone.

Convert AVI to iPhone MP4

ffmpeg -i [source].avi -f mp4 -vcodec mpeg4 -b 250000 -s 480?320 -acodec aac -ar 24000 -ab 64 -ac 2 [destination].mp4

for 4:3 aspect:

ffmpeg -i source-xvid.avi -s 480x320 -aspect 4:3 -b 768k -ab 64k -ar 22050 -r 30000/1001 OUT.mp4

for 16:9:

ffmpeg -i source-xvid.avi -s 480x320 -aspect 16:9 -b 768k -ab 64k -ar 22050 -r 30000/1001 OUT.mp4

Create a video that is supported by youtube:

ffmpeg -i mymovie.mpg -ar 22050 -acodec libmp3lame -ab 32K -r 25 -s 320x240 -vcodec flv
mytarget.flv

Takes an mpeg video and coverts it to a youtube compatible flv file.
The -r 25 sets the frame rate for PAL, for NTSC use 29.97

4. Audio Conversion

Convert RM file to mp3

ffmpeg -i input.rm -acodec libmp3lame -ab 96k output.mp3

Adjust the bitrate (-ab) as necessary. If omitted FFmpeg will use a default of 64 kb/s.

Converting WMV to MP3 using FFMPEG

ffmpeg -i audio1.wmv audio1.mp3

This will convert audio1.wmv file to audio1.mp3
Converting WMV to FLV using FFMPEG

ffmpeg -i audio1.wmv audio1.flv

This will convert audio1.wmv file to audio1.flv, this will generate only audio content
Converting AMR to MP3 using FFMPEG

ffmpeg -i audio1.amr -ar 22050 audio1.mp3

This will convert audio1.amr file to audio1.mp3 having audio rate 22.05 Khz
Converting aac to mp3 using FFMPEG

ffmpeg -i audio1.aac -ar 22050 -ab 32 audio1.mp3

This will convert audio1.aac to audio1.mp3 having audio rate 22.05 Khz and Audio BitRate 32Khz
Converting aac to mp3 using FFMPEG with MetaData

ffmpeg -i audio1.aac -ar 22050 -ab 32 -map_meta_data audio1.mp3:audio1.aac audio1.mp3

This will convert audio1.aac to audio1.mp3 having audio rate 22.05 Khz and Audio BitRate 32Khz and will copy the meta data from .aac file to .mp3 file

5. Audio Extraction

ffmpeg -i video.avi -f mp3 audio.mp3

Dumping Audio stream from flv (using ffmpeg)

ffmpeg -i input.flv -f mp3 -vn -acodec copy ouput.mp3

6. Record Audio and Video from webcam

To record video run ffmpeg with arguments such as these:

ffmpeg -f video4linux2 -s 320x240 -i /dev/video0 out.mpg

To record both audio and video run ffmpeg with arguments such as these:

ffmpeg -f oss -i /dev/dsp -f video4linux2 -s 320x240 -i /dev/video0 out.mpg

7. Copy Only A Part Of Video

Cut out a piece of film from a file. Choose an arbitrary length and starting time.

ffmpeg -vcodec copy -acodec copy -i orginalfile -ss 00:01:30 -t 0:0:20 newfile

-vcodec, you choose what video codec the new file should be encoded with. Run ffmpeg -formats E to list all available video and audio encoders and file formats.

copy, you choose the video encoder that just copies the file.

-acodec, you choose what audio codec the new file should be encoded with.

copy, you choose the audio encoder that just copies the file.

-i originalfile, you provide the filename of the original file to ffmpeg

-ss 00:01:30, you choose the starting time on the original file in this case 1 min and 30 seconds into the film

-t 0:0:20, you choose the length of the new film

newfile, you choose the name of the file created.

8. Join Multiple Video Files

A few multimedia containers (MPEG-1, MPEG-2 PS, DV) allow to join video files by merely concatenating them.

Hence you may concatenate your multimedia files by first transcoding them to these privileged formats, then using the humble cat command (or the equally humble copy under Windows), and finally transcoding back to your format of choice.

mkfifo orig1.mpg
mkfifo orig2.mpg
ffmpeg -i input1.avi -sameq -y orig1.mpg
ffmpeg -i input2.avi -sameq -y orig2.mpg

Merge files

cat orig1.mpg orig2.mpg | ffmpeg -f mpeg -i - -vcodec copy -acodec copy merged.mpg

Merge and convert to avi

cat orig1.mpg orig2.mpg | ffmpeg -f mpeg -i - -sameq -vcodec mpeg4 -acodec libmp3lame merged.avi

Notice that you should either use -sameq or set a reasonably high bitrate for your intermediate and output files, if you want to preserve video quality.

Also notice that you may avoid the huge intermediate files by taking advantage of named pipes, should your platform support it:

9. Removing Synchronization Problems Between Audio and Video

ffmpeg -i source_audio.mp3 -itsoffset 00:00:10.2 -i source_video.m2v target_video.flv

This assumes that there is a 10.2 sec delay between the video and the audio (delayed).

To extract the original video into a audio and video composites look at the command on extracting audio and video from a movie

Here is more information of how to use ffmpeg:
http://www.ffmpeg.org/ffmpeg-doc.html