Computer Scientist

Saturday, 19 November 2011

Use LD_PRELOAD to load another version library before running

LD_PRELOAD is a fantastic method to debug the code or library.

Basically, this just record other's work on the LD_PRELOAD and description. That's is fair enough, so I won't bored to modify that:


You Are Here

LD_PRELOAD fun

Here is a welcome digression from my previous Twitter oriented posts. I’m starting to play around with the LD_PRELOAD feature in the Linux dynamic linker. For those who might not know what this feature is, here is the description from ld.so (8).

LD_PRELOAD
              A whitespace-separated list of additional,  user-specified,  ELF
              shared  libraries  to  be loaded before all others.  This can be
              used  to  selectively  override  functions   in   other   shared
              libraries.   For  setuid/setgid  ELF binaries, only libraries in
              the standard search directories that are  also  setgid  will  be
              loaded.
So in pratical term, any libraries you specify in the LD_PRELOAD environment variable will loaded before any system libraries. This means that dynamic symbols in a loading program will be first searched in those libraries before being searched anywhere else. This means you can override any defined symbol you want in standard libraries.
Let’s start with a rather juvenile example. This will change the behavior of the read (2) function in order to make the user believe a file might have a different content.

 
ssize_t read(int fd, void *buf, size_t count) {
    static int done = 0;
    if (!done) {
        char silly_str[] = "Haha you got overriden.\n";
        size_t s = count > sizeof(silly_str) ? sizeof(silly_str) : count;
        memcpy(buf, silly_str, s);
        done = 1;
        return s;
    }
    else return 0;
}
If you compile this inside a library that is called, for example, libread.so, you can test this code by running:
> /bin/cat /etc/fstab
# /etc/fstab: static file system information.
#
...
> LD_LIBRARY_PATH=. LD_PRELOAD=libread.so /bin/cat /etc/fstab
Haha you got overriden.
That in itself is just a rather silly prank you can play on your friend’s computer if you happen to have access to it. Experienced programmer will start seeing potential uses for LD_PRELOAD. I am getting to that.
The subject of our next example will be the honorable ls (1). ls uses the opendir (3) function to open a directory and browse its files. It should react properly if it can’t open the directory. One way to test this is to make opendir() return NULL and observe how the caller reacts. You can do that using LD_PRELOAD.

DIR *opendir(const char *name) {
    return NULL;
}
 
> LD_LIBRARY_PATH=. LD_PRELOAD=libls1.so /bin/ls /tmp
/bin/ls: cannot open directory /tmp
What can you do now if you want to preserve part of the behavior of the function, or modify they result it returns? Your preloaded library will then need to use libdl to dynamically load the function it wants to modify the behavior.
The following example is a very simple override of the opendir (3) function which open a different directory than what the caller expects. I will explain more in detail the details of this function below.

DIR *opendir(const char *name) {
    DIR *(*libc_opendir)(const char *name);
    *(void **)(&libc_opendir) = dlsym(RTLD_NEXT, "opendir");
    return libc_opendir("/tmp");
}

libdl is fortunately very simple to use. The naive approach would be to use dlopen (3) to open the C library, then get the pointer to the function you are calling using dlsym (3). In theory, this technique is valid and working, but doing that circumvents the LD_PRELOAD mechanisme because preloaded libraries can be chained and calling directly into the C library prevents other caller to override our own function.
In practice, calling dlopen() on libc on an Ubuntu Karmic system made some program crash and burn for reasons I will not attempt to explain. The next technique should be preferred on Linux system, especially when dealing with the system C library.
dlsym() has an option that makes the Linux dynamic linker search for the right symbol to be override. This is the RTLD_NEXT flag, which is to be used just for the purpose of wrapper dynamic library functions.
libdl the task of returning the pointer to the right symbol. The RTLD_NEXT option to dlsym() returns the right symbol.
The next and final example of the use of LD_PRELOAD will still use the valiant ls. In time for Christmas, this will modify the output of ls by randomizing the d_type field returned in the dirent structure by readdir (3). If you use colorized ls output, and I believe most of you probably do, you should see a pretty display of color whenever you list a directory by preloading this function.

struct dirent64 *readdir64(DIR *dir) {
    static struct dirent64 *(* libc_readdir64)(DIR *dir) = NULL;
    struct dirent64 *dent;
    unsigned char rnd_dtype[7] = { DT_UNKNOWN, DT_REG,
                                   DT_DIR, DT_FIFO,
                                   DT_SOCK, DT_CHR,
                                   DT_BLK };
    if (libc_readdir64 == NULL) {
        *(void **)(&libc_readdir64) = dlsym(RTLD_NEXT, "readdir64");
        srand(time(NULL));
    }
    dent = libc_readdir64(dir);
    if (dent != NULL)
        dent->d_type = rnd_dtype[rand() % 7];
    return dent;
}
There is still a problem with this code on my new Ubuntu Hardy machine. The code from the preloaded library hangs before the program terminates. I do not understand why this happen and a search for this bug did not turn up anything. The problem doesn’t happen with Ubuntu Karmic.
There is nothing new about using LD_PRELOAD this way. Several very nice libraries have been built with the intention of modifying the behavior of typical libraries.
  • fakeroot: “fakeroot provides a fake root environment by means of LD_PRELOAD and SYSV IPC (or TCP) trickery.”
  • fakechroot: fakechroot provides a fake chroot environment to programs.
  • libtrash:“[...] the shared library which, when preloaded, implements a trash can under GNU/Linux”
  • cowdancer: cowdancer is an userland implementation of copy-on-write filesystem.
There are 29 projects matching LD_PRELOAD on freshmeat.net. You might have used some of them.
The code I have written for this demonstration is available on BitBucket.
Written by fdgonthier
January 11th, 2010 at 10:10 pm
Posted in Debian,Linux,Programming,Tips and Tricks
Tagged with , , , ,

reference:  http://www.lostwebsite.net/2010/01/ld_preload-fun/

Libcage Change Log

1. cage.*pp dump network CF_EXP for logging
2. udphandler.*pp dump network CF_EXP logging
3. dht.*pp dump dht storage CF_EXP logging
4. add elog into libcage to logging
5. add random library to get random number and poisson number.
6. add cf_exp, cf1_exp, cf2_exp for myself experiment.

TODO:
  • migrate to Cmake from Omake(It seems that hopeless)
  • using clock_gettime() to replace gettimeofday()

Timing problem in LInux

Gettimeofday???
Linux provides a 'gettimeofday()' function for users to check the epoch time. However, it gets the system's best guess at wall time. This can go backwards. This is why a 'timer_correct' function is embedded in the Libevent's event.c file. This function will be used when the "MONOTONIC" clock is not in used.


Monotonic clock
In Linux, another function 'clock_gettime(CLOCK_MONOTONIC)' is used to obtain the monotonic time, where the monotonic means that there is no possible to get a time backwards with this function. In this case, this is more reasonable used be used.

"POSIX.1-2008 marks gettimeofday() as obsolete, recommending the use of clock_gettime(2) instead."

Wednesday, 5 October 2011

Allocate memory in a function

I used to try to find some ways to allocate memory in a function. When other functions invoke this function with a null pointer as a parameter, this null pointer would be filled with some contents.

The most silly way to accomplish this task is like this:

void fill_function (void * pointer){
    pointer = (void *) malloc (certain length);

    pointer is filled with some contects;
}

void main () {
    int * file_content;
    fill_function(file_content);

    printf("", file_content);
}

Here, the most important thing which is ignored is the malloc function will allocate the memory in a certain position of the memory which is completely decided by malloc not the one on the left side of the equal symbol. If the file_content pointer is initialized with a number 5(assume that this is a memory address). The fill_function will not allocate the memory space from 5. The malloc will find another free place (for example 102, another memory address) and allocate the continuous memory space after 102, the local pointer variable pointer will be set to be 102. However, the invoking function main has no chance to catch this allocated address.

So a more reasonable way is like this:

int * fill_function (){
    int * pointer = (int *) malloc (certain length);

    return pointer;
}

void main(){
    int * file_content = fill_function();

    printf("", file_content);
}

Another problem is the garbage allocation collection problem. At this point, the boost::shared_ptr<> is encouraged to be used in this situation if you are programming by C++.

Tuesday, 28 June 2011

Sorting mechanism in Java (Refer to other's work)

Java Sorting: Comparator vs Comparable Tutorial

Java Comparators and Comparables? What are they? How do we use them? This is a question we received from one of our readers. This article will discuss the java.util.Comparator and java.lang.Comparable in details with aset of sample codes for further clarifications.

Prerequisites

  • Basic Java knowledge

System Requirements


What are Java Comparators and Comparables?

As both names suggest (and you may have guessed), these are used for comparing objects in Java. Using these concepts; Java objects can be
sorted according to a predefined order.

Two of these concepts can be explained as follows.

Comparable

A comparable object is capable of comparing itself with another object. The class itself must implements the java.lang.Comparable interface in order to be able to compare its instances.

Comparator

A comparator object is capable of comparing two different objects. The class is not comparing its instances, but some other class’s instances. This comparator class must implement the java.util.Comparator interface.

Do we need to compare objects?

The simplest answer is yes. When there is a list of objects, ordering these objects into different orders becomes a must in some situations. For example; think of displaying a list of employee objects in a web page. Generally employees may be displayed by sorting them using the employee id. Also there will be requirements to sort them according to the name or age as well. In these situations both these (above defined) concepts will become handy.

How to use these?

There are two interfaces in Java to support these concepts, and each of these has one method to be implemented by user.
Those are;

java.lang.Comparable: int compareTo(Object o1)
This method compares this object with o1 object. Returned int value has the following meanings.
  1. positive – this object is greater than o1
  2. zero – this object equals to o1
  3. negative – this object is less than o1

java.util.Comparator: int compare(Object o1, Objecto2)
This method compares o1 and o2 objects. Returned int value has the following meanings.
  1. positive – o1 is greater than o2
  2. zero – o1 equals to o2
  3. negative – o1 is less than o1

java.util.Collections.sort(List) and java.util.Arrays.sort(Object[]) methods can be used to sort using natural ordering of objects.
java.util.Collections.sort(List, Comparator) and java.util.Arrays.sort(Object[], Comparator) methods can be used if a Comparator is available for comparison.

The above explained Employee example is a good candidate for explaining these two concepts. First we’ll write a simple Java bean to represent the Employee.

public class Employee {
    private int empId;
    private String name;
    private int age;

    public Employee(int empId, String name, int age) {
        // set values on attributes
    }
    // getters & setters
}

Next we’ll create a list of Employees for using in different sorting requirements. Employees are added to a List without any specific order in the following class.

import java.util.*;

public class Util {
    
    public static List<Employee> getEmployees() {
        
        List<Employee> col = new ArrayList<Employee>();
        
        col.add(new Employee(5, "Frank", 28));
        col.add(new Employee(1, "Jorge", 19));
        col.add(new Employee(6, "Bill", 34));
        col.add(new Employee(3, "Michel", 10));
        col.add(new Employee(7, "Simpson", 8));
        col.add(new Employee(4, "Clerk",16 ));
        col.add(new Employee(8, "Lee", 40));
        col.add(new Employee(2, "Mark", 30));
        
        return col;
    }
}

Sorting in natural ordering

Employee’s natural ordering would be done according to the employee id. For that, above Employee class must be altered to add the comparing ability as follows.

public class Employee implements Comparable<Employee> {
    private int empId;
    private String name;
    private int age;
    
    /**
     * Compare a given Employee with this object.
     * If employee id of this object is 
     * greater than the received object,
     * then this object is greater than the other.
     */
    public int compareTo(Employee o) {
        return this.empId - o.empId ;
    }
    ….
}

The new compareTo() method does the trick of implementing the natural ordering of the instances. So if a collection of Employee objects is sorted using Collections.sort(List) method; sorting happens according to the ordering done inside this method.

We’ll write a class to test this natural ordering mechanism. Following class use the Collections.sort(List) method to sort the given list in natural order.

import java.util.*;

public class TestEmployeeSort {
    
    public static void main(String[] args) {     
        List coll = Util.getEmployees();
        Collections.sort(coll); // sort method
        printList(coll);
    }
    
    private static void printList(List<Employee> list) {
        System.out.println("EmpId\tName\tAge");
        for (Employee e: list) {
            System.out.println(e.getEmpId() + "\t" + e.getName() + "\t" + e.getAge());
        }
    }
}

Run the above class and examine the output. It will be as follows. As you can see, the list is sorted correctly using the employee id. As empId is an int value, the employee instances are ordered so that the int values ordered from 1 to 8.

EmpId Name Age
1 Jorge 19
2 Mark 30
3 Michel 10
4 Clerk 16
5 Frank 28
6 Bill 34
7 Simp 8
8 Lee 40

Sorting by other fields

If we need to sort using other fields of the employee, we’ll have to change the Employee class’s compareTo() method to use those fields. But then we’ll loose this empId based sorting mechanism. This is not a good alternative if we need to sort using different fields at different occasions. But no need to worry; Comparator is there to save us.

By writing a class that implements the java.util.Comparator interface, you can sort Employees using any field as you wish even without touching the Employee class itself; Employee class does not need to implement java.lang.Comparable or java.util.Comparator interface.

Sorting by name field

Following EmpSortByName class is used to sort Employee instances according to the name field. In this class, inside the compare() method sorting mechanism is implemented. In compare() method we get two Employee instances and we have to return which object is greater.

public class EmpSortByName implements Comparator<Employee>{

    public int compare(Employee o1, Employee o2) {
        return o1.getName().compareTo(o2.getName());
    }
}

Watch out: Here, String class’s compareTo() method is used in comparing the name fields (which are Strings).

Now to test this sorting mechanism, you must use the Collections.sort(List, Comparator) method instead of Collections.sort(List) method. Now change the TestEmployeeSort class as follows. See how the EmpSortByName comparator is used inside sort method.

import java.util.*;

public class TestEmployeeSort {
    
    public static void main(String[] args) {
        
        List coll = Util.getEmployees();
        //Collections.sort(coll);
        //use Comparator implementation
        Collections.sort(coll, new EmpSortByName());
        printList(coll);
    }
    
    private static void printList(List<Employee> list) {
        System.out.println("EmpId\tName\tAge");
        for (Employee e: list) {
            System.out.println(e.getEmpId() + "\t" + e.getName() + "\t" + e.getAge());
        }
    }
}

Now the result would be as follows. Check whether the employees are sorted correctly by the name String field. You’ll see that these are sorted alphabetically.

EmpId Name Age
6 Bill 34
4 Clerk 16
5 Frank 28
1 Jorge 19
8 Lee 40
2 Mark 30
3 Michel 10
7 Simp 8

Sorting by empId field

Even the ordering by empId (previously done using Comparable) can be implemented using Comparator; following class
does that.

public class EmpSortByEmpId implements Comparator<Employee>{

    public int compare(Employee o1, Employee o2) {
        return o1.getEmpId() - o2.getEmpId();
    }
}

Explore further

Do not stop here. Work on the followings by yourselves and sharpen knowledge on these concepts.
  1. Sort employees using name, age, empId in this order (ie: when names are equal, try age and then next empId)
  2. Explore how & why equals() method and compare()/compareTo() methods must be consistence.

If you have any issues on these concepts; please add those in the comments section and we’ll get back to you.


Reference : http://lkamal.blogspot.com/2008/07/java-sorting-comparator-vs-comparable.html

Monday, 13 June 2011

Double pointer's usages (on going)

The first usage of the double pointer is to allocate memory for a given pointer:

         nlp = &res.readdir_res_u.list;
while (d = readdir(dirp)) {
nl = *nlp = (namenode *)malloc(sizeof(namenode));
nl->name = strdup(d->d_name);
nlp = &nl->pNext;
}
*nlp = NULL;

This comes from "Power Programming with RPC P84"

Sunday, 6 March 2011

Mean, Median, Mode (Averages) and Range

How to find a common statistical number is a crucial part of post-experiment data analysis. However, too many notations would pop into our mind if we start to carry out the data analysis. I searched across the whole Internet and tried to find all of the useful notations in this field. And I will also extend to other formal literatures in the future.

Mean: add up all of the numbers and divide by how many numbers there are.

Median: put the numbers in order and find the middle number.

Mode: the most common number.

Range: the difference between the biggest number and the smallest number.

Thursday, 24 February 2011

Usefule Linux Commands:

Kill all of the specific process:

killall -9 <command name>

This is different a little from UNIX, BE CAREFUL!!

Saturday, 19 February 2011

Found a Bug in jBittorrentAPI

Actually, this is my first time to find some bugs in other's work.....

In it's TorrentProcessor class, it get the value of file length using a Long.inValue. The precision has been lost a lot, even if it assign it back to a long value. Because of this bug, I got a overflowed number with minus to be inserted into mysql's unsigned big int column!! Obviously, a error message is obtained.

Actually, the reason that I will use this package is that, it can read all of the files that C version's bencode program can not read. Whenever I process the files from Pirate Bay, there would be a segmentation fault sent back to me.

Good work for jBittorrentAPI.....

Change table properties in MySQL::

The most basic one is :


ALTER TABLE person MODIFY person_id SMALLINT UNSIGNED AUTO_INCREMENT;



More detailed can be found here:
http://dev.mysql.com/doc/refman/5.1/en/alter-table.html

What if you encounter an error message of "out of heap" in Java?

At this time, you need to:
 (1). check the program and find out any possibility to free the unnecessary resource.
 (2). increase your heap size to achieve more memory allocated to java.

Here I only care about the later one:

java -Xms<initial heap size> -Xmx<maximum heap size>
Defaults are:
java -Xms32m -Xmx128m


                                    

Usage of u_int32_t and size_t

Actually, the usage of these kinds of types confused me for a quite long time, because I really don't know what's the differences between these types and the C's primitive types such as 'int'?

The C's primitive types such as 'long', 'int' are machine dependent, which means different types system probably has different definition of long and int. For example, in old x86 (486), 'int' may be defined by 2 bytes. However, it is 4 bytes in most machines nowadays. In this case, 'u_int32_t' or 'int32_t' is invented to be a system independent type. In fedora, they are defined in file '/usr/include/sys/types.h', 'u_int32_t' is only  possible to be a 'int' type with 32bit (4 bytes). This will provide the program with being portable.

Friday, 18 February 2011

Experiment Q&A............

When I compiled the lookup program using MySQL connector C++ statically:

Q: what if I am confronted with "ld: can not found lm "?
A: The reason is that 'ld' can not find the static version of 'libm.a' within the default position.
     So I need to install it in Fedora using yum: "sudo yum install glibc-static". This should solve the 'lm' problem.

Q: what if I am confronted with message: "undefined reference to 'pthread_self' and so on...."?
A: The problem is that the pthread library can not be found. Just direct the g++ the location of it by append "-lpthread" at the end.

Q: What is the command that I used to compile the program and link against the static library?
A: Like this: "g++ -static -o <dst> <src> /usr/local/lib/libmysqlcppconn-static.a /usr/local/mysql/lib/libmysqlclient.a -lpthread".

Q: When using the static library, sometimes there is a strange message pop out: can not found lc ?
A: Because the static version of glibc is not installed by default, we need to install by ourself:
        sudo yum install glibc-static


Thursday, 17 February 2011

DELETE all of rows in a TABLE in MySQL (Refer to others' work)

Delete and Truncate

There are two ways to delete all the data in a MySQL database table.
TRUNCATE TABLE tablename; This will delete all data in the table very quickly. In MySQL the table is actually dropped and recreated, hence the speed of the query. The number of deleted rows for MyISAM tables returned is zero; for INNODB it returns the actual number deleted.
DELETE FROM tablename; This also deletes all the data in the table, but is not as quick as using the "TRUNCATE TABLE" method. In MySQL >= 4.0 the number of rows deleted is returned; in MySQL 3.23 the number returned is always zero.

Auto Increment Columns for MyISAM Tables

If you have an auto increment primary key column in your MyISAM table the result will be slightly different depending which delete method you use. When using the "TRUNCATE TABLE" method the auto increment seed value will be reset back to 1. When using the "DELETE FROM" method the auto increment seed will be left as it was before (eg if the auto increment field of last inserted record was 123 the next inserted record will be set to 124).
Note that this is true for MySQL >= 4.0; from my reading of the TRUNCATE manual page in MySQL 3.23 TRUNCATE works just like DELETE which would mean the auto increment seed is not reset. I do not currently have a 3.23 database set up to test it so cannot confirm this.

Auto Increment Columns for INNODB Tables

For INNODB tables, whether you use the "TRUNCATE TABLE" or "DELETE FROM" methods, the auto increment field will not be reset. If you inserted 5 records into a new table, then deleted all records and inserted another record, the field would have a value of 6, regardless of which method you used.
Update 17 Feb 2009: I originally wrote this post when MySQL 4.0 was the current version. I've just tested the above now on an INNODB table using MySQL 5.0 and using TRUNCATE does reset the auto increment field back to the default. So either the behaviour changed at some point or I was incorrect when making the above statement.

Are you really sure you want to delete all data?

Before deleting all the data in a database you should make sure you really intend to delete all the data. It often pays first to "SELECT * FROM tablename" or "SELECT COUNT(*) FROM tablename" before doing so to check that it really is safe to delete all data. Maybe you really want to do something like "DELETE FROM tablename WHERE foo = 'bar'" instead.

Reference: http://www.electrictoolbox.com/article/mysql/delete-all-data-mysql/

Monday, 7 February 2011

Touch files in a directory recursively

Using the following command is able to touch files recursively:

find . -print0 | xargs -r0 touch

where . is the current directory and the option r of xargs is specific for GNU xargs.






Monday, 17 January 2011

Bash Script-exec

The Open Group Base Specifications Issue 6
IEEE Std 1003.1, 2004 Edition
Copyright © 2001-2004 The IEEE and The Open Group, All Rights reserved.

NAME

exec - execute commands and open, close, or copy file descriptors

SYNOPSIS

exec [command [argument ...]]

DESCRIPTION

The exec utility shall open, close, and/or copy file descriptors as specified by any redirections as part of the command.
If exec is specified without command or arguments, and any file descriptors with numbers greater than 2 are opened with associated redirection statements, it is unspecified whether those file descriptors remain open when the shell invokes another utility. Scripts concerned that child shells could misuse open file descriptors can always close them explicitly, as shown in one of the following examples.
If exec is specified with command, it shall replace the shell with command without creating a new process. If arguments are specified, they shall be arguments to command. Redirection affects the current shell execution environment.

OPTIONS

None.

OPERANDS

See the DESCRIPTION.

STDIN

Not used.

INPUT FILES

None.

ENVIRONMENT VARIABLES

None.

ASYNCHRONOUS EVENTS

Default.

STDOUT

Not used.

STDERR

The standard error shall be used only for diagnostic messages.

OUTPUT FILES

None.

EXTENDED DESCRIPTION

None.

EXIT STATUS

If command is specified, exec shall not return to the shell; rather, the exit status of the process shall be the exit status of the program implementing command, which overlaid the shell. If command is not found, the exit status shall be 127. If command is found, but it is not an executable utility, the exit status shall be 126. If a redirection error occurs (see Consequences of Shell Errors ), the shell shall exit with a value in the range 1-125. Otherwise, exec shall return a zero exit status.

CONSEQUENCES OF ERRORS

Default.

The following sections are informative.

APPLICATION USAGE

None.

EXAMPLES

Open readfile as file descriptor 3 for reading:
exec 3< readfile
Open writefile as file descriptor 4 for writing:
exec 4> writefile
Make file descriptor 5 a copy of file descriptor 0:
exec 5<&0
Close file descriptor 3:
exec 3<&-
Cat the file maggie by replacing the current shell with the cat utility:
exec cat maggie

RATIONALE

Most historical implementations were not conformant in that:
foo=bar exec cmd
did not pass foo to cmd.

FUTURE DIRECTIONS

None.

SEE ALSO

Special Built-In Utilities

CHANGE HISTORY

Issue 6

IEEE Std 1003.1-2001/Cor 1-2002, item XCU/TC1/D6/5 is applied so that the reference page sections use terms as described in the Utility Description Defaults ( Utility Description Defaults ). No change in behavior is intended.
End of informative text.
 
Refer to: http://pubs.opengroup.org/onlinepubs/009695399/utilities/exec.html

Saturday, 15 January 2011

Linux Network Security Issues

Until now, as far as I know, there are at least three different levels of network security mechanism that affects the running of network processes, TCP Wrapper, Iptables firewall, SELinux.

TCP Wrapper and SELinux are host-based mechanism. According to a response to a network packet, TCP Wrapper and SELinux will decide if this packet can be processed by the specific running process. They will not block any network access from other hosts, however, they will limit the running process in the host from processing network messages.

Iptables, on the other hand, provides a network-based security mechanism. It inspects every network packet whenever a packet going into a host or leaving a host. A great amount of distinct functional rules can be set up in order to filter some specific packets. By this way, unwanted packet is forbidden outside of the host.

In my recent experiment, several problems are suffered in these two issues. I'd like to record them here for future reference:

  1. In school's Fedora 11 system, snmp messages can not going out even if the corresponding port is opened by iptables firewall. When I was using tcpdump to inspect every packet of snmp protocol, I found that snmp request messages were able to go in the system. but there were never any packets coming out. I supposed that two possibilities: snmp crashed, or some other than iptables was keeping block the outgoing messages. Finally, after I searched almost the whole internet (joking, can I? but it is true that it is quite difficult to spot a specific rear problem on the internet.), I realised that I got half correct. There is something called TCP Wrapper which is used by Linux system to prevent some specific daemon processes from accessing from unwanted network hosts. In this case, all of the process other than several processes denoted in the file /etc/hosts.allow is allowed to be accessed in localhost host which means that only local access is allowed. This is right the reason why I was able to query snmp using localhost. In coming snmp messages from other hosts is not allowed to be processed by snmpd (so harsh!!). The solution is simple, just add the snmpd into hosts.allow file. 
  2. The problem regarding to Iptables is a little foolish, but I learnt others when I modified the rules of Iptables. Actually, the reason of I can't transfer files to cspc020 is because I open the wrong port for tcp connection (don't believe the instructions on the webpage totally, this lessoned me). Just open tcp60000 is fine. During this process, i found that, always modify the iptables using iptables command line tools before modify the configuration file /etc/sysconfig/iptables, because the command line is temporary but effects at once. If there are some problems, I am able to resume it by restart computer and then the original configuration will be read. !!!! Good mechanism.


Thursday, 13 January 2011

Strange SNMP problem in Ubuntu or other platform

Description of the problem:
After I have installed SNMPd in Ubuntu 9.04, I was trying to snmpwalk system to test the correction of the installation and deployment. However, I got a Time out message even though I did not get any firewall installed in Ubuntu. At the beginning, I thought that it is the firewall problem like in Fedora. The truth of no firewall installed in Ubuntu by default makes me realise that this is not in that case.

Solution:
Ubuntu is different from Fedora, there is a file to keep the default snmpd's running options, which is:
/etc/default/snmpd. In it, the options denote that only local host is able to snmpwalk the SNMP agent. In this case, I will not only need to change the snmpd.conf configure file, but I will also need to change some lines in the snmpd configure file mentioned above to make it work. The changing is like this:


#SNMPDOPTS='-Lsd -Lf /dev/null -u snmp -I -smux -p /var/run/snmpd.pid 127.0.0.1'
SNMPDOPTS='-Lsd -Lf /dev/null -u snmp -I -smux -p /var/run/snmpd.pid -c /etc/snmp/snmpd.conf