Showing posts with label Linux. Show all posts
Showing posts with label Linux. Show all posts

Wednesday, September 26, 2012

GParted to resize your existing hard drive

I am using VMware as my Virtual Disk but it is the same as standalone or dualboot machines.
First you need to ISO image I am using linuxmint-13-cinnamon-dvd-64bit.iso which can be downloaded from Linux Mint. You cannot resize the hard driver unless you load from LiveCD or ISO image. i.e.
kubuntu-14.04-desktop-amd64.iso. VMware setting and expand the hard drive you want it.

You need to hit F2 to go to BIOS setup and setup CD as 1st boot. In VMware there are 2 variables on VMX file locates in your Virtual Machines -> Name of  your Virtual Machine.VMX.
If these variables are exist you can add as follow to let VMware knows. The string name are exactly as you see.
bios.bootDelay = "5000"
bios.forceSetupOnce = "TRUE"
The 1st option delay boot time so you have enough time to hit F2 and 2nd option it boots into BIOS or ESC to choose the device to boot up. This is one time boot selection.

This is BIOS of VMware and not BIOS of the system. VMware creates a BIOS similar to BIOS system to handle Boot and other things. VMware using PhoenixBIOS. My system BIOS is ACER.
To distinguish between 2 BIOS. I call system BIOS and VMware BIOS.
When load into this VMware BIOS you need to change CD/DVD file points to ISO that you want to run.
With the virtual machine powered off, open the Configuration Editor (choose Settings then Configuration Editor), and select the DVD/CD-ROM. If not, it won't boot from this iso even you hit ESC to choose CD/DVD boot. This is a virtual CD/DVD drive not physical CD drive !!!. no need to put CD in the physical drive if choose ISO option instead of physical drive. The default of ISO image is pointing to C:/Program Files(X86)/VMware Player/linux.iso all you need to point this to right ISO.
Using edit VMware Hardware then Click on ISO image instead of physical CD/DVD. My case is  linuxmint-13-cinnamon-dvd-64bit.iso the iso just downloaded into Downloads directory.
In this CD/DVD setup choose use ISO to boot and device status select connect at power on so it can boot.
 Remember the new size is only good to VMware but Linux-Mint won't know until you change the size using GParted. YOU CANNOT RESIZE IF IT BOOTS FROM THE SAME HARD DRIVE for example /dev/sda1 if you drive to resize it will prompt that device is busy it needs to umount first and this is the reason we boot from LiveCS ISO image. After it boots we can go change the hard drive of /dev/sda1 because it is umounted.
When ISO boot up choose demo and run from there. Do not select installation all we want is to use Gparted.
If GParted in not installed in this live CD need to install: sudo apt-get install gparted. The ISO is stored from disk or USB, the demo runs from RAM so gparted in run from RAM when it is shut down gparted is gone and it does not write into ISO file. Since the disk expand is rarely using so to issue the command sudo apt-get install gparted is a small thing.

Ref.
http://communities.vmware.com/message/1061677 
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=102



This is system BIOS. It is different with VMware BIOS above.
If you have a problem to install Win 7 with RAID then Disable RAID in BIOS and enabled native IDE.
RAID (redundant array of independent disks)




Thursday, July 19, 2012

linux Kernel 2.6.38 using sema_init

I just found out when upgraded rt73 device drivers that the old Linux kernel and the new version they changed the mutex to semaphore with extra parameter (A mutex is essentially the same thing as a binary semaphore(1 or 0 single resource) vs. counting values that allows multiple program threads to share the same resource)  I could not even grep for the name because the name is totally different. The name is now all low cases sema_init. So the device driver could fail after upgrade to a new Linux kernel and this is the case we need to change the device driver rt73 module rtmp_init.c as follow.
#ifndef init_MUTEX_LOCKED

        sema_init(&(pAd->usbdev_semaphore), 1);            // linux kernel 2.6.38

        sema_init(&(pAd->mlme_semaphore), 1);

        sema_init(&(pAd->RTUSBCmd_semaphore), 1);

#else
        init_MUTEX_LOCKED(&(pAd->usbdev_semaphore)); // linux kernel 2.6.37

        init_MUTEX_LOCKED(&(pAd->mlme_semaphore));

        init_MUTEX_LOCKED(&(pAd->RTUSBCmd_semaphore));  


#endif
//Note:

//currently typically to 1 (in which case the semaphore is called a mutex) 
//From Linus Torvalds
/* Maximum concurrent users 
#define MAX_CONCURRENT_USERS 20
sema_init(&sem, MAX_CONCURRENT_USERS);

However, almost all practical use of semaphores is a special case where
the counter is initialized to 1, and where they are used as simple
mutual exclusion with only one user allowed in the critical region.
Such a semaphore is often called a "mutex" semaphore for MUTual
EXclusion.
*/

Friday, June 08, 2012

Processes Vs. Threads revisited

http://objectlayer.blogspot.ca/2006/05/thread-vs-process.html
- All threads in a program must run the same executable.A child process, on the
other hand, may run a different executable by calling an exec function.
- An errant thread can harm other threads in the same process because threads
share the same virtual memory space and other resources. For instance, a wild
memory write through an uninitialized pointer in one thread can corrupt
memory visible to another thread.
An errant process, on the other hand, cannot do so because each process has a
copy of the program’s memory space.
- Copying memory for a new process adds an additional performance overhead
relative to creating a new thread. However, the copy is performed only when
the memory is changed, so the penalty is minimal if the child process only reads
memory.
- Threads should be used for programs that need fine-grained parallelism. For
example, if a problem can be broken into multiple, nearly identical tasks, threads
may be a good choice. Processes should be used for programs that need coarser
parallelism.
- Sharing data among threads is trivial because threads share the same memory.
(However, great care must be taken to avoid race conditions, as described previously.)
Sharing data among processes requires the use of IPC mechanisms, as
described in Chapter 5.This can be more cumbersome but makes multiple
processes less likely to suffer from concurrency bugs.
- Thread has its own Code, Data, Stack and Thread is sharing. Each Process has one or more threads and each thread belongs to one process
- Thread is fast for context switch and cheaper than process.
Conceptually, a thread exists within a process.Threads
Simple process for ls

#include <pthread.h>
int main()
{
  int return_value;
  return_value = system ("ls -l /");
  return return_value;
}



#include <pthread.h>
#include <stdio.h>
/* Prints x’s to stderr.*/
void* print_xs (void* unused)
{
    while (1)
    fputc ('x', stderr);
    return NULL;
}
/* The main program.
*/
int main ()
{
    pthread_t thread_id;
    /* Create a new thread. The new thread will run the print_xs
    function. */
    pthread_create (&thread_id, NULL, &print_xs, NULL);
    /* Print o’s continuously to stderr. */
    while (1)
    fputc ('o', stderr);
    return 0;
}
Compile and link this program using the following code:
$ gcc -o thread-create thread-create.c -lpthread
$ ./thread-create
ooooxxxxxxxxxxxxxxooooo

Is linux kernel preemptive or not?

Yes, the kernel is preemptive.
It has been preemptive by default since the 2.6 branch. The old Linux kernel is not preemptive, which means that a process can be preempted only while running in User Mode; non preemptive kernel design is much simpler, since most synchronization problems involving the kernel data structures are easily avoided Prior to Linux kernel version 2.5.4, Linux Kernel was not preemptive which means a process running in kernel mode cannot be moved out of processor until it itself leaves the processor or it starts waiting for some input output operation to get complete.
  Generally a process in user mode can enter into kernel mode using system calls. Previously when the kernel was non-preemptive, a lower priority process could priority invert a higher priority process by denying it access to the processor by repeatedly calling system calls and remaining in the kernel mode. Even if the lower priority process' timeslice expired, it would continue running until it completed its work in the kernel or voluntarily relinquished control. If the higher priority process waiting to run is a text editor in which the user is typing or an MP3 player ready to refill its audio buffer, the result is poor interactive performance. This way non-preemptive kernel was a major drawback at that time.
A preemptive kernel is one that can be interrupted in the middle of a executing code - for instance in response for a system call - to do other things and run other threads, possible that are not in the kernel.
The main advantage in a preemptive kernel is that sys-calls do not block the entire system. if a sys-call takes a long time to finish then it doesn't mean the kernel can't do anything else in this time.
The main disadvantage is that this introduces more complexity to the kernel code, having to handle more end-cases, perform more fine grained locking or use lock-less structures and algorithms.
preemption (more correctly pre-emption) is the act of temporarily interrupting a task being carried out by a computer system, without requiring its cooperation, and with the intention of resuming the task at a later time. Such a change is known as a context switch. It is normally carried out by a privileged task or part of the system known as a preemptive scheduler, which has the power to preempt, or interrupt, and later resume, other tasks in the system.
$ strace echo test                                
execve("/bin/echo", ["echo", "test"], [/* 45 vars */]) = 0       (execve, mmap are system calls)         
brk(0)                                  = 0xc90000  
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f4210db7000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)         
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f4210db5000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f4210c5d000
write(1, "test\n", 5test
)                   = 5
close(1)                                = 0
munmap(0x7f4210c5d000, 4096)            = 0
close(2)                                = 0
exit_group(0)                           = ?

Kernel space and user space


System calls are used when a user space program wants to use some data or some service provided by the Linux kernel. In current Linux kernel (2.6.23) there exist 324 system calls. All system calls provide a function which is useful for many application programs (such as file operations, network operations or process related operations). There is no point in adding a very specific system call which can be used only by a specific program. http://www.kernel.org/doc/man-pages/online/pages/man2/mknod.2.html
A conventional computer operating system usually segregates virtual memory into kernel space and user space. Kernel space is strictly reserved for running the kernel, kernel extensions, and most device drivers. In contrast, user space is the memory area where all user mode applications work and this memory can be swapped out when necessary.
Similarly, the term userland refers to all application software that runs in user space.[1] Userland usually refers to the various programs and libraries that the operating system uses to interact with the kernel: software that performs input/output, manipulates file system objects, etc.
Each user space process normally runs in its own virtual memory space, and, unless explicitly requested, cannot access the memory of other processes. This is the basis for memory protection in today's mainstream operating systems, and a building block for privilege separation. Depending on the privileges, processes can request the kernel to map part of another process's memory space to its own, as is the case for debuggers. Programs can also request shared memory regions with other processes.
Kernel space and user space is the separation of the privileged operating system functions and the restricted user applications. The separation is necessary to prevent user applications from ransacking your computer. It would be a bad thing if any old user program could start writing random data to your hard drive or read memory from another user program's memory space.
User space programs cannot access system resources directly so access is handled on the program's behalf by the operating system kernel. The user space programs typically make such requests of the operating system through system calls.


Write a device driver in Linux

/* Necessary includes for device drivers http://www.freesoftwaremagazine.com/articles/drivers_linux?page=0%2C1 */

#include <linux init.h>

//#include <linux config.h> no more since 2.6.19 deprecated

#include <linux module.h>

#include <linux kernel.h> /* printk() */

#include <linux slab.h> /* kmalloc() */

#include <linux fs.h> /* everything... */

#include <linux errno.h> /* error codes */

#include <linux types.h> /* size_t */

#include <linux proc_fs.h>

#include <linux fcntl.h> /* O_ACCMODE */

#include <asm system.h> /* cli(), *_flags */

#include <asm uaccess.h> /* copy_from/to_user */



MODULE_LICENSE("Dual BSD/GPL");



/* Declaration of memory.c functions */

int memory_open(struct inode *inode, struct file *filp);

int memory_release(struct inode *inode, struct file *filp);

ssize_t memory_read(struct file *filp, char *buf, size_t count, loff_t *f_pos);

ssize_t memory_write(struct file *filp, char *buf, size_t count, loff_t *f_pos);

void memory_exit(void);

int memory_init(void);



/* Structure that declares the usual file */

/* access functions */

struct file_operations memory_fops = {

  read: memory_read,

  write: memory_write,

  open: memory_open,

  release: memory_release

};



/* Declaration of the init and exit functions */

module_init(memory_init);

module_exit(memory_exit);



/* Global variables of the driver */

/* Major number */

int memory_major = 60;

/* Buffer to store data */

char *memory_buffer;



int memory_init(void) {

  int result;



  /* Registering device */

  result = register_chrdev(memory_major, "memory", &memory_fops);

  if (result < 0) {

    printk(

      "<1>memory: cannot obtain major number %d\n", memory_major);

    return result;

  }



  /* Allocating memory for the buffer */

  memory_buffer = kmalloc(1, GFP_KERNEL); 

  if (!memory_buffer) { 

    result = -ENOMEM;

    goto fail; 

  } 

  memset(memory_buffer, 0, 1);



  printk("<1>Inserting memory module\n"); 

  return 0;



  fail: 

    memory_exit(); 

    return result;

}



void memory_exit(void) {

  /* Freeing the major number */

  unregister_chrdev(memory_major, "memory");



  /* Freeing buffer memory */

  if (memory_buffer) {

    kfree(memory_buffer);

  }



  printk("<1>Removing memory module\n");



}



int memory_open(struct inode *inode, struct file *filp) {



  /* Success */

  return 0;

}

int memory_release(struct inode *inode, struct file *filp) {



  /* Success */

  return 0;

}

ssize_t memory_read(struct file *filp, char *buf, 

                    size_t count, loff_t *f_pos) { 



  /* Transfering data to user space */ 

  copy_to_user(buf,memory_buffer,1);



  /* Changing reading position as best suits */ 

  if (*f_pos == 0) { 

    *f_pos+=1; 

    return 1; 

  } else { 

    return 0; 

  }

}

ssize_t memory_write( struct file *filp, char *buf,

                      size_t count, loff_t *f_pos) {



  char *tmp;



  tmp=buf+count-1;

  copy_from_user(memory_buffer,tmp,1);

  return 1;

}

///----Makefile
obj-m = hello.o memory.o
KVERSION = $(shell uname -r)
all:
    make -C /lib/modules/$(KVERSION)/build M=$(PWD) modules
clean:
    make -C /lib/modules/$(KVERSION)/build M=$(PWD) clean
///----Testing after make
$sudo nsmod memory.ko
$sudo rmmod memory.ko
$ dmesg | less
[223447.209899] Inserting memory module
[223629.779878] Removing memory module
[223663.099549] Loading hello module...
# mknod /dev/memory c 60 0 (make device using system call see http://www.kernel.org/doc/man-pages/online/pages/man2/mknod.2.html  In the above, c means that a char device is to be created, 60 is the major number and 0 is the minor number.)
Insert character into the device
#chmod 666 /dev/memory
#insmod memory.ko
#echo -n ABC > /dev/memory
#cat /dev/memory
C
Note: .ko kernel object,.so /usr/lib (library), .ro (module plugin gedit)
.o for object then link gcc -o test test.c

Questions:
Q. Can you explain me what is device files and how do I access or see device files? Why UNIX / Linux has device files?
A. Under Linux and UNIX each and every hardware device treated as a file. A device file allows to accesses hardware devices so that end users do not need to get technical details about hardware.
In short, a device file (also called as a special file) is an interface for a device driver that appears in a file system as if it were an ordinary file. This allows software to interact with the device driver using standard input/output system calls, which simplifies many tasks.

Device file two types

There are two types of device files based upon how data written to them and read from them is processed by the operating system and hardware:
  • Character special files or Character devices
  • Block special files or Block devices

Understanding Character special files or Character devices

  • Talks to devices in a character by character (1 byte at a time)
  • Examples: Virtual terminals, terminals and serial modems etc

Understanding Block special files or Block devices

  • Talks to devices 1 block at a time ( 1 block = 512 bytes to 32KB)
  • Examples: Hard disk, DVD/CD ROM, and memory regions etc

Why use device files?

Device file allows transparent communication between user space applications and computer hardware.

What is a Zombie process?

http://en.wikipedia.org/wiki/Zombie_process On Unix and Unix-like computer operating systems, a zombie process or defunct process is a process that has completed execution but still has an entry in the process table. This entry is still needed to allow the parent process to read its child's exit status. The term zombie process derives from the common definition of zombie — an undead person. In the term's metaphor, the child process has "died" but has not yet been "reaped". Also, unlike normal processes, the kill command has no effect on a zombie process.

How do I kill zombie process?

You cannot kill zombies, as they are already dead. But if you have too many zombies then kill parent process or restart service.
You can kill zombie process using PID obtained from any one of the above command. For example kill zombie process having PID 4104:
# kill -9 4104
create zombie.c

#include <stdlib.h&g

#include <sys types.h&g

#include <unistd.h&g



int main ()

{

pid_t child_pid;



child_pid = fork (); // system call to create a new process referred to as the child process

if (child_pid > 0) {

sleep (60);

}

else {

exit (0);                  // parent exist make child becomes zombie if the parent crashes makes it becomes

}                             // orphan process will waste resources but zombie is no harm except running out PID

return 0;

}

gcc zombie.c -o zombie
./zombie
after 1 minute
ps -ax
  1815 ?        Sl     0:00 /usr/bin/python /usr/bin/bzr-notify
 6314 pts/1    S+     0:00 ./zombie
 6315 pts/1    Z+     0:00 [zombie]
The child process is marked as , and its status code is Z, for zombie.
When the program exits, the child process is inherited by init. This process should cleans up the zombie proces automatically.

Zombie vs. Orphan process

http://en.wikipedia.org/wiki/Zombie_process
On Unix and Unix-like computer operating systems, a zombie process or defunct process is a process that has completed execution but still has an entry in the process table. This entry is still needed to allow the parent process to read its child's exit status. The term zombie process derives from the common definition of zombie — an undead person. In the term's metaphor, the child process has "died" but has not yet been "reaped". Also, unlike normal processes, the kill command has no effect on a zombie process.
A zombie process is not the same as an orphan process. An orphan process is a process that is still executing, but whose parent has died. They do not become zombie processes; instead, they are adopted by init (process ID 1), which waits on its children.

Saturday, May 20, 2006

XAMPP for apache, php and mysql

Download: http://www.apachefriends.org/en/xampp.html

XAMPP for Linux
The distribution for Linux systems (tested for SuSE, RedHat, Mandrake and Debian) contains: Apache, MySQL, PHP & PEAR, Perl, ProFTPD, phpMyAdmin, OpenSSL, GD, Freetype2, libjpeg, libpng, gdbm, zlib, expat, Sablotron, libxml, Ming, Webalizer, pdf class, ncurses, mod_perl, FreeTDS, gettext, mcrypt, mhash, eAccelerator, SQLite and IMAP C-Client.
XAMPP for Windows
The distribution for Windows 98, NT, 2000 and XP. This version contains: Apache, MySQL, PHP + PEAR, Perl, mod_php, mod_perl, mod_ssl, OpenSSL, phpMyAdmin, Webalizer, Mercury Mail Transport System for Win32 and NetWare Systems v3.32, JpGraph, FileZilla FTP Server, mcrypt, eAccelerator, SQLite, and WEB-DAV + mod_auth_mysql.

Tuesday, May 16, 2006

ActiveX vs. Java Applet

ActiveX is a technology developed by Microsoft. With an ActiveX-enabled browser (ie Internet Explorer only) ActiveX controls can be downloaded as part of a Web document to add functionality to the browser (similar to Java applets). In particular ActiveX enables seamless viewing of Windows files of all types, eg spreadsheets, and in combination with other technologies such as Java and scripting languages, makes possible the development of complex Web applications. ...

Does Active X works with Linux, I guess it is not because it is not windows, I tried Java applets for dialing to Netzero (ISP), it is all works for both linux and windows. Linux needs jre-1_5_0-linux-i586.bin and windows need C:\Program Files\Java\jre1.5.0_01\bin\npjpi150_01.dll

Linux:
download netzero.deb
cp netzero.deb /
cd /
ar -xv netzero.deb
gunzip data.tar.gz
tar xvf data.tar
Install java jre
cd /opt/nzclient
./runclient.sh

Create a file to get on desktop of cd ~/gnome-desktop links/associated to /java/jre-1_5_0/opt/runclient.sh
Click to dial from desktop of gnome.

Thread vs. process

One difference: Threads within a process share a single address space, and can, with care (synchronization), access one another's variables. Processes run in different address spaces, and must use inter-process communications mechanisms to exchange information.
Processes Versus Threads


Other Terms:

Heavyweight Process = Process
Lightweight Process = Thread


Advantages (Thread vs. Process):

Much quicker to create a thread than a process.
Much quicker to switch between threads than to switch between processes.
Threads share data easily


Disadvantages (Thread vs. Process):

No security between threads: One thread can stomp on another thread's data.
For threads which are supported by user thread package instead of the kernel:
If one thread blocks, all threads in task block

Thread States (Example Java):

New = New: Placed on the ready list.
Blocked = Wait for an event
Runnable = Ready or Running
Dead = Exit


Thread functions: POSIX (Standard UNIX C/C++):

pthread_create(): creates a thread
pthread_exit(): kills itself
pthread_kill(): sends a signal to a specified thread
pthread_join(): waits for a thread to exit.
pthread_self(): returns thread id
synchronization functions


Java Implementation
http://objectlayer.blogspot.ca/2012/06/java-threads-for-hello-world.html
Define a class to contain a thread, by defining:
class Worker1 extends Thread OR
class Worker2 implements Runnable (Recommended)
The parent thread creates the object and invokes its start() function.
The start function creates the child thread and calls the object’s run() function.
Thread terminates when thread returns from run(), or stop() is called.

Types of Thread Implementations


User Thread Package

A package outside the OS creates and schedules threads.
Model: Many-to-one: Many user threads associated with one process.
Thread creation/management is faster than Kernel Thread support
However if one thread blocks, then entire process blocks. Thus all threads block.


Kernel Thread Support

The OS creates and schedules threads.
Model: One-to-one: Each user thread is associated with one kernel thread.
If one thread blocks, the kernel can schedule other threads.
If multiprocessor configuration, kernel can allocate threads of same process on different processors.
Example: Windows NT, Solaris, Digital UNIX

Many-to-Many Model: Combination of above

N user threads share M kernel threads (where N > M)
There may be a flexible and changing assignment of user threads to kernel threads.
Bound: User-thread is assigned permanently to kernel thread
Unbound: User threads share available kernel threads
Example: Solaris
----------------------deadlock vs. race condition
DeadlocksA deadlock occurs when one or more threads are permanently blocked from executing because each thread waits on a resource held by another thread in the deadlock. A thread can also deadlock on itself. The simplest case of deadlock occurs when process p1 gains access to data structure a and process p2 gains access to b, but p1 then waits for b and p2 waits for a.
A deadlock
occurs when one or more threads are stuck waiting for something that never will
occur.(advanced linux program p.82)
Race Conditions
A race condition occurs when two or more threads perform an operation, and the result of the operation depends on unpredictable timing factors; specifically, when each thread executes and waits and when each thread completes the operation.
The result is trouble. Because both processes see the same scull device, each will
store its new memory in the same place in the quantum set. If A stores its pointer
first, B will overwrite that pointer when it does its store. Thus the memory allocated by A, and the data written therein, will be lost.