Who is Who [who is who] [news] [archives] [staff] [forum] [friends] [DOWNLOAD]
_Supernodes
Table of Content:
- Overview
- Supernode vs HP Virtualize
- CyN Development
- Kernel Basis (HURD and menuetOS based)
- Clusterin Analisys
- Sphread Information Analisis Against Server Farm Platform
- Security Analisys Against Honeypots Research
- Portability And Stability
- Conlusion
- Bibliography
- Overview
The biggest problem in the most “supernode” architectures is that the functionalities of clients and servers are strictly separated
or dependients of the processor vectoring analisys, clients can not carry workloads on servers even in case servers are very busy.
P2P supernode architecture for example comes to another extreme, system workloads are uniformly distributed on all peers without
the consideration of heterogeneous resources among peers. Each node is supposed to make equal contribution. This seems natural
since equal treatment of peers is the basic principle of P2P system. But unfortunately, computer resources, such as network bandwidth,
computation power and storage capacity are quite different among peers. Different peers are treated differently and make unequal
contributions according to their capabilities. System workloads are neither uniformly distributed on all nodes nor concentrated on a
small number of centralized servers, they are unevenly spread. Supernodes take most system workloads and act more like servers
while ordinary nodes only take a small portion of system workloads and act more like clients.The apporach in two cases are so difficult to bring to the practice, Supernode selection is a fundamental problem that appears
throughout P2P because the vectorying work is not efficient in the most cases, However I assume for the fact, first made a best
mathematical analisys become on matrix game.
We reconfigure the network factors like a complementary combinatory formula.
Generating step by step the network schema using transition model:
Now finally we have the model based on taylor expansion (for timestamp between node model)
Example of relationships on nodes:
The result based on neuronal networks:
The neuron
Between each timestamp on each “nodes” we have a “time gain” for each process and the main centralized node can remember
each time elapsed for each process requested, the next time our schema it not comsume so much time in request from system
calls or system resources. Basically it can be a “neuron” or kernel for neuron integrated on that.The mathematics for this intro is poor is not proof here, but is’nt the pourpose of this paper, only I develate the principal proofs
of the network supernode math theory.
A node can be a supernode only if certain criteria are satisfied. The basic requirement is a high bandwidth connection. This
property is important to gain low network latency in largescale P2P systems. The second criteria is it must has enough computation
power because supernodes need to deal with most system workloads. Third, supernodes can not join/leave system frequently,
otherwise its efficiency will be greatly reduced. Also it is helpful if supernodes have large amount of storage space.For this schema, our schema, we don’t need the workload of a “basis supernode” defun, we factorize the second criteria with
“scalable neuron network kernel integrated based” to computes anyway tasks on demand against the workloads stress cases…The main problem on a standard supernode design are because the overheads of creating a new supernode domain, supernodes
must be stable. A frequently joining/leaving node can not be a supernode even though it has a high bandwidth connection.
Otherwise, the system maintenance overhead will increase greatly and the performance gain will decrease sharply. To encourage
a node to apply to be a supernode, some benefits should be given, for example, large storage space quota can be assigned to
the client who contributes its node as a supernode. An adversary supernode can bring great trouble to system performance.
Security techniques such as credit system, quota limitation must be performed to relieve the problem, more researches are needed
in supernode security. The “no centralize” schema are more capable if the centralize kernel share himself to the network node to
supernode ca. We asumes all network have the same capabilities on processing (hardware) and space memory, but the centralice
node (supernode first schema) can be more robust and may contain the first copy and installation of kernel.So, we need a special Sistem Operating features? The answer is simple:
The most distributions of SO on the network and communities don’t support our supernode schema, the principal vendors:
microsoft, novell, sun and siemens are not designed for work with our mathematical and structured schema. We need a new
design for do the schema functionally at 100%.
We need based on P2P scheme of supernodes with the nodes array only, the operating mode is different, also the kernel modules
(including neuronal modules on that).When, Routing/locating is the most frequent executed operation in P2P systems.
How we work with these nodes become and fixing with our kernel mode?
Improvements in routing/locating algorithm can greatly boost system overall performance. Current routing algorithms simply
treat nodes as if they have the same amount of computer resources and can deal with workloads equally. However, a node
with a fast CPU and a broad bandwidth connection (power node) can deal with a routing task much faster than a node with
a slow CPU and a narrow bandwidth connection (weak node). Pastry uses a smart mechanism to relieve this problem. When
a new node comes, all the entries in its routing table are created by choosing a node which is topologically close to the current
node. However, it has limitations. If the number of system nodes is small, a weak node can not be avoided if it is the only node
satisfying the prefix requirement. Basically our schema solve this problem using the neuronal integrated modules, basically we
“connect” all stations (and news also) like a new neurons who uses the principal kernel as a live copy on it, we integrated a
“worm” who execute a automata routine for detects new station connected on the network system and ask first if the protocol
and system kernel is prepared (with a simple mark on guest connected system) , if the host respond yes, the centralize node
try to install on it the mechanism of live non booting kernel copy like a beowoolf clustering systems has be.In those moment the system mapping the new station and include her capacity on the internal table and updating all stations
tables for “a new host exist and are started”…
On the next items for this papers we explain how the internal kernel operates and made it beautiful task.
- Supernode vs Virtualize
Virtualize
The concept behind server virtualization is not new, however. IBM has been creating virtual machines on its mainframes since the 1960s
So what is it? Virtualization breaks the link between the hardware and the applications that run on it. This includes virtual storage,
virtual networking, application virtualization and, the focus of this tutorial, server virtualization. It requires the installation of a
software layer that allows more than one server to operate on the same piece of hardware. There are two basic approaches to
this:1. The one popularized by VMware (now part of EMC Hopkinton, Mass.) runs a virtualization layer (called a hypervisor) between
the hardware and the operating system. With this method, several operating systems can run on the same set of hardware.
The drawback is that each virtual server requires its own operating system, which adds to licensing costs and system overhead.2. Sun Microsystems, of Santa Clara, Calif., takes the opposite approach. It installs its Solaris operating system directly on
the hardware. Different applications run in isolated areas called "containers," but they all share the same operating system
instance. Shares of the physical server resources are then assigned to each of the containers on a permanent or dynamic basis.There are many reasons for adopting server virtualization. A popular one is better resource utilization. It is not uncommon to
see servers running at 10 percent or less of their capacity, at different points in the day. By letting several virtual servers share
a single set of hardware, a much higher average utilization rate is achieved, and hardware and support costs are lowered.Virtualization also makes it easier to provision and reallocate servers. Instead of having to manually set up a server, the
virtualization software can set up a server using a pre-existing template and shift server images from one physical server
to another to balance workloads or improve efficiency. It can also automatically set up a new virtual server on a different
machine when there is a hardware malfunction. Each application is isolated from the others, which provides greater security.AMD and Intel have both included virtualization support in their chips. In AMD's case, this involves taking some of the
commands that normally would be handled by the Virtual Machine Manager and including them in the chips' instruction sets.
Similarly, Intel has released its Intel Virtualization Technology (IVT) for desktops, Xeon server and Itanium server CPUs.Utilities, too, are starting to support virtual servers. Backing up virtual servers, for example, poses a series of unique challenges.
One of the biggest challenges involves server consolidation. The advantage of server consolidation is that all the virtual servers
share the same set of hardware, which works out great when all servers are running at low levels. But backup is a resource
intensive activity in terms of disk I/Os, processor utilization and traffic through the network interface card. Backup vendors,
therefore, have developed a number of techniques for backing up virtual servers. CommVault Systems offers customers the
option of backing up the entire virtual server into a single large file, and Syncsort's Backup Express can either backup each
virtual server individually or backup the entire physical server.In addition, management tool vendors are including virtual server support in their products. TeamQuest Corporation of Clear Lake,
Iowa, for example, supports virtualization throughout its product line. It has a set of five collection agents for VMware ESX Server
2.0, which gather information both on the performance of the virtual machines (e.g., CPU, disk, memory and NIC) and on the ESX
service console (e.g., disk space, process-workload and system log messages). Reports and alarms can be set up on each of the
individual virtual servers or on the physical server. TeamQuest's capacity planning software can model each virtual machine as an
individual workload as well as determine the impact of running several virtual machines on a physical server.The primary action in setting up a virtual server is selecting and installing the virtualization layer. Here are some of the more
popular options.
- Xen 3.0: Xen is a lightweight open source hypervisor (less than 50,000 lines of code) which runs on Intel or AMD x86
and 64-bit processors, with or without virtualization technologies. It supports up to 32-way SMP (Simultaneous Multi
Processing) and requires a modification of the client operating system, which means it will run Linux but not Windows clients.
Although the original Xen hypervisor works only with Linux clients, XenSource, the company behind the Xen project, released
XenEnterprise, a version that supports Windows Server and Solaris guests as well.
- Windows Virtual Server 2005 R2: Microsoft initially charged for its virtualization technology, and it was limited to Windows
servers. With Windows Server 2003R2, customers can run up to four operating systems on a physical server. On April 3,
Microsoft announced it was making Virtual Server a free download, and it extended support to clients running nine versions
of Red Hat and SUSE Linux.
- VMware Server: VMware (EMC) is by far the largest vendor of virtualization technology for x86 platforms. In early 2006,
the company released VMware Server, a replacement for GSX Server, which is a single server virtualization platform for
Linux and Windows. More than 100,000 downloads of this free product were made in the first week alone. VMware Server has
all the features of the GSX Server, and adds support for virtual SMP, Intel Virtualization Technology and 64-bit guest operating
systems.
- VMware ESX Server: Although its entry-level product is now free, VMware still charges for its enterprise-class ESX Server.
ESX server runs on x86-based servers and supports Linux (Red Hat and SUSE), Windows (Server and XP), Novell NetWare and
FreeBSD 4.9 clients.
- Virtual Iron: Virtual Iron is another company offering Xen-based products. It has four products: two free single server
versions, an enterprise version and one for clusters. In addition to the Xen hypervisor, Virtual Iron also includes management
tools and an administrative interface.
- IBM Virtualization Engine Platform: This platform encompasses the entire line of IBM servers. As well as the usual
hypervisor for server partitioning, it includes virtual I/O and virtual Ethernet, a workload manager and management console.
- SWSoft Virtuozzo: Virtuozzo takes an approach similar to that of Solaris. It runs above, rather than below, the operating
systems. It has two versions — one for Windows and another for Linux — and customers can create virtual servers on top of
these. One particular application is for running "Virtual Private Servers (VPS)" or a hosting facility. With Virtuozzo, a single
physical server can run up to 5,000 VPSes.
Although the basic concept of virtualization is likely to be around for quite a while, it is not clear whether virtualization software
will always be a separate product. IBM, EMC, Microsoft, AMD, Intel and others are incorporating greater features into their product
line, and this trend will continue. Overall, though, virtualization appears to be moving toward being the default method of server
operation, rather than being just for a specific niche. Eventually, this may make discrete hypervisors a thing of the past.
Supernode
A supernode is the controller of the system. It is a special node which handles all requests from other nodes for joining or
leaving a session. Its task is to maintain an accurate global view of the topology of a session. A supernode can handle multiple
sessions, i.e. multiple topologies. Kernel running on the supernode is responsible for producing a list of affected nodes and
the corresponding actions when a new node joins or leaves the network. The supernodes then use the outputs from the kernel
to send out appropriate messages to other nodes to instruct them with appropriate actions.Nodes Case
The node wanting to host the session will contact the SuperNode and provides its information such as the streaming file,
and the prefered transport protocol. Assuming that the supernode can control another session, it creates a new Session entry.
Creating a session entry mainly involve creating a new kernel image on it, which maintains a new session. Next, the SuperNode
sends to the host a Session ID. Now the session is listed on the SuperNode and any node wanting to join a session will contact it.Whenever a node wants to join an existing session it contacts the SuperNode and get the list of sessions managed by the
SuperNode. It then picks the session and sends a Join request along with the Session ID to the Supernode. When the Supernode
receives a join message from a peer, depending on the session ID, the corresponding kernel for that session updates the topology
to reflect the new node. Updating the topology has two aspects. Firstly, existing nodes have to be updated so that they can find
their new neighbors. Secondly, the new nodes need to know about the nodes it needs to connect to. The kernel does the work of
identifying the nodes that need updating and passes on the information to the SuperNode. The SuperNode then sends Update messages
to these nodes instructing them about the change. When the change is completed, the peers send back messages confirming the success,
and the kernel updates the new topology. We note that when a node joins the network, only a small number of nodes are affected,
and therefore the hybrid architecture is highly scalable.When a peer wants to leave a session, it will inform the concerned Supernode of its intent. The Supernode then passes the peer's
session ID to the corresponding kernel ID on the general table. The kernel on centralize supernode then deletes the node and provides
the supernode network a list of affected nodes. Next, the Supernode sends out messages to these nodes informing them of the changes.
Once the affected nodes make their new connections, they inform the Supernode and the Supernode accepts the leave request.
This process ensures that the leaving node does not cause disruption to the streaming by creating temporary disconnections. Because
a node may leave the network without informing the Supernode, all the neighbors periodically sending heartbeats to each other. If a
node does not receive a heartbeat from its neighbor for a specific period of time, it informs the Supernode about its neighbor leaving
the network and appropriate action is taken as described above.Advantages
- Live run/installation: Since the design separates the work of handling the topology management, data dissemination and
process distribution, the peer becomes very simple. All it has to do is to forward the data packets as instructed by the
Supernode. The complexity of running the algorithm and any issues related to optimizing the performance of session are
handled by the Supernode.
- Security: Having centralized control over who joins and leaves the system prevents the malicious users from entering
the network. The Supernode can personally authenticate the nodes participating in a separate kernel session over the
principal node called centralize supernode.
- Flexibility: Simple architecture offers high flexibility in terms of management and upgradability. Suppose, we wish to
upgrade the kernel on centralize supernode, or change the network topology, or add more security/controlling features,
we just need to change the kernel on run time in the centralize Supernode. The rest of the system can remain almost
the same. This is because the Supernode sends to the peers the standard messages and the peers simply follows the
instructions contained in the standard messages. In addition, the system has the advantage that any change need not
be known to the peers because all the control is handled by the centralize Supernode. Thus the whole system is easy to
manage and upgrade for automatically by the centralize who running again the worm agent to upgrades each kernel
session copy…3. CyN Development
CyN (Converting your Node) is a operating system model based on linux operating systems. It is based on HURD and menuetOS
work schemes with an adaptative kernel for isolating work and reassembly his own cached structure depends the needs of each
host instalation. The original idea for create cyn is the capability of adaptation and include neuronal algorithm design specially
for network clustering and supernode infrastructure.We need start with a kernel internals and system background first:
The kernel is the "core" of any computer system: it is the "software" which allows users to share computer resources.
The kernel can be thought as the main software of the OS (Operating System), which may also include graphics management.
For example, under Linux (like other Unix-like OSs), the XWindow environment doesn't belong to the Linux Kernel, because it manages only graphical operations (it uses user mode I/O to access video card devices).
By contrast, Windows environments (Win9x, WinME, WinNT, Win2K, WinXP, and so on) are a mix between a graphical
environment and kernel.
What is the difference between User Mode and Kernel Mode?
Many years ago, when computers were as big as a room, users ran their applications with much difficulty and, sometimes, their
applications crashed the computer.Operative modes
To avoid having applications that constantly crashed, newer OSs were designed with 2 different operative modes:
Kernel Mode: the machine operates with critical data structure, direct hardware (IN/OUT or memory mapped), direct memory, IRQ,
DMA, and so on.
User Mode: users can run applications
| Applications /|\| ______________ || | User Mode | || ______________ || | |Implementation | _______ _______ | AbstractionDetail | | Kernel Mode | || _______________ || | || | || | |\|/ Hardware |
Kernel Mode "prevents" User Mode applications from damaging the system or its features.
Modern microprocessors implement in hardware at least 2 different states. For example under Intel, 4 states determine the
PL (Privilege Level). It is possible to use 0,1,2,3 states, with 0 used in Kernel Mode.
Unix OS requires only 2 privilege levels, and we will use such a paradigm as point of reference.
Switching from User Mode to Kernel Mod
When do we switch?
Once we understand that there are 2 different modes, we have to know when we switch from one to the other.
Typically, there are 2 points of switching:
- When calling a System Call: after calling a System Call, the task voluntary calls pieces of code living in Kernel Mode
- When an IRQ (or exception) comes: after the IRQ an IRQ handler (or exception handler) is called, then control returns
back to the task that was interrupted like nothing was happened.System Calls
System calls are like special functions that manage OS routines which live in Kernel Mode.
A system call can be called when we:
- access an I/O device or a file (like read or write)
- need to access privileged information (like pid, changing scheduling policy or other information)
- need to change execution context (like forking or executing some other application)
- need to execute a particular command (like ''chdir'', ''kill", ''brk'', or ''signal'')
| |------->| System Call i | (AccessingDevices)| | | | [sys_read()] || ... | | | || system_call(i) |-------- | || [read()] | | || ... | | || system_call(j) |-------- | || [get_pid()] | | | || ... | ------->| System Call j | (Accessing kernel data structures)| | | [sys_getpid()]|| |USER MODE KERNEL MODEUnix System Calls WorkingSystem calls are almost the only interface used by User Mode to talk with low level resources (hardware). The only exception
to this statement is when a process uses ''ioperm'' system call. In this case a device can be accessed directly by User Mode
process (IRQs cannot be used).
NOTE: Not every ''C'' function is a system call, only some of them.
Below is a list of System Calls under Linux Kernel 2.4.17, from [ arch/i386/kernel/entry.S ].long SYMBOL_NAME(sys_ni_syscall) /* 0 - old "setup()" system call*/.long SYMBOL_NAME(sys_exit).long SYMBOL_NAME(sys_fork).long SYMBOL_NAME(sys_read).long SYMBOL_NAME(sys_write).long SYMBOL_NAME(sys_open) /* 5 */.long SYMBOL_NAME(sys_close).long SYMBOL_NAME(sys_waitpid).long SYMBOL_NAME(sys_creat).long SYMBOL_NAME(sys_link).long SYMBOL_NAME(sys_unlink) /* 10 */.long SYMBOL_NAME(sys_execve).long SYMBOL_NAME(sys_chdir).long SYMBOL_NAME(sys_time).long SYMBOL_NAME(sys_mknod).long SYMBOL_NAME(sys_chmod) /* 15 */.long SYMBOL_NAME(sys_lchown16).long SYMBOL_NAME(sys_ni_syscall) /* old break syscall holder */.long SYMBOL_NAME(sys_stat).long SYMBOL_NAME(sys_lseek).long SYMBOL_NAME(sys_getpid) /* 20 */.long SYMBOL_NAME(sys_mount).long SYMBOL_NAME(sys_oldumount).long SYMBOL_NAME(sys_setuid16).long SYMBOL_NAME(sys_getuid16).long SYMBOL_NAME(sys_stime) /* 25 */.long SYMBOL_NAME(sys_ptrace).long SYMBOL_NAME(sys_alarm).long SYMBOL_NAME(sys_fstat).long SYMBOL_NAME(sys_pause).long SYMBOL_NAME(sys_utime) /* 30 */.long SYMBOL_NAME(sys_ni_syscall) /* old stty syscall holder */.long SYMBOL_NAME(sys_ni_syscall) /* old gtty syscall holder */.long SYMBOL_NAME(sys_access).long SYMBOL_NAME(sys_nice).long SYMBOL_NAME(sys_ni_syscall) /* 35 */ /* old ftime syscall holder */.long SYMBOL_NAME(sys_sync).long SYMBOL_NAME(sys_kill).long SYMBOL_NAME(sys_rename).long SYMBOL_NAME(sys_mkdir).long SYMBOL_NAME(sys_rmdir) /* 40 */.long SYMBOL_NAME(sys_dup).long SYMBOL_NAME(sys_pipe).long SYMBOL_NAME(sys_times).long SYMBOL_NAME(sys_ni_syscall) /* old prof syscall holder */.long SYMBOL_NAME(sys_brk) /* 45 */.long SYMBOL_NAME(sys_setgid16).long SYMBOL_NAME(sys_getgid16).long SYMBOL_NAME(sys_signal).long SYMBOL_NAME(sys_geteuid16).long SYMBOL_NAME(sys_getegid16) /* 50 */.long SYMBOL_NAME(sys_acct).long SYMBOL_NAME(sys_umount) /* recycled never used phys() */.long SYMBOL_NAME(sys_ni_syscall) /* old lock syscall holder */.long SYMBOL_NAME(sys_ioctl).long SYMBOL_NAME(sys_fcntl) /* 55 */.long SYMBOL_NAME(sys_ni_syscall) /* old mpx syscall holder */.long SYMBOL_NAME(sys_setpgid).long SYMBOL_NAME(sys_ni_syscall) /* old ulimit syscall holder */.long SYMBOL_NAME(sys_olduname).long SYMBOL_NAME(sys_umask) /* 60 */.long SYMBOL_NAME(sys_chroot).long SYMBOL_NAME(sys_ustat).long SYMBOL_NAME(sys_dup2).long SYMBOL_NAME(sys_getppid).long SYMBOL_NAME(sys_getpgrp) /* 65 */.long SYMBOL_NAME(sys_setsid).long SYMBOL_NAME(sys_sigaction).long SYMBOL_NAME(sys_sgetmask).long SYMBOL_NAME(sys_ssetmask).long SYMBOL_NAME(sys_setreuid16) /* 70 */.long SYMBOL_NAME(sys_setregid16).long SYMBOL_NAME(sys_sigsuspend).long SYMBOL_NAME(sys_sigpending).long SYMBOL_NAME(sys_sethostname).long SYMBOL_NAME(sys_setrlimit) /* 75 */.long SYMBOL_NAME(sys_old_getrlimit).long SYMBOL_NAME(sys_getrusage).long SYMBOL_NAME(sys_gettimeofday).long SYMBOL_NAME(sys_settimeofday).long SYMBOL_NAME(sys_getgroups16) /* 80 */.long SYMBOL_NAME(sys_setgroups16).long SYMBOL_NAME(old_select).long SYMBOL_NAME(sys_symlink).long SYMBOL_NAME(sys_lstat).long SYMBOL_NAME(sys_readlink) /* 85 */.long SYMBOL_NAME(sys_uselib).long SYMBOL_NAME(sys_swapon).long SYMBOL_NAME(sys_reboot).long SYMBOL_NAME(old_readdir).long SYMBOL_NAME(old_mmap) /* 90 */.long SYMBOL_NAME(sys_munmap).long SYMBOL_NAME(sys_truncate).long SYMBOL_NAME(sys_ftruncate).long SYMBOL_NAME(sys_fchmod).long SYMBOL_NAME(sys_fchown16) /* 95 */.long SYMBOL_NAME(sys_getpriority).long SYMBOL_NAME(sys_setpriority).long SYMBOL_NAME(sys_ni_syscall) /* old profil syscall holder */.long SYMBOL_NAME(sys_statfs).long SYMBOL_NAME(sys_fstatfs) /* 100 */.long SYMBOL_NAME(sys_ioperm).long SYMBOL_NAME(sys_socketcall).long SYMBOL_NAME(sys_syslog).long SYMBOL_NAME(sys_setitimer).long SYMBOL_NAME(sys_getitimer) /* 105 */.long SYMBOL_NAME(sys_newstat).long SYMBOL_NAME(sys_newlstat).long SYMBOL_NAME(sys_newfstat).long SYMBOL_NAME(sys_uname).long SYMBOL_NAME(sys_iopl) /* 110 */.long SYMBOL_NAME(sys_vhangup).long SYMBOL_NAME(sys_ni_syscall) /* old "idle" system call */.long SYMBOL_NAME(sys_vm86old).long SYMBOL_NAME(sys_wait4).long SYMBOL_NAME(sys_swapoff) /* 115 */.long SYMBOL_NAME(sys_sysinfo).long SYMBOL_NAME(sys_ipc).long SYMBOL_NAME(sys_fsync).long SYMBOL_NAME(sys_sigreturn).long SYMBOL_NAME(sys_clone) /* 120 */.long SYMBOL_NAME(sys_setdomainname).long SYMBOL_NAME(sys_newuname).long SYMBOL_NAME(sys_modify_ldt).long SYMBOL_NAME(sys_adjtimex).long SYMBOL_NAME(sys_mprotect) /* 125 */.long SYMBOL_NAME(sys_sigprocmask).long SYMBOL_NAME(sys_create_module).long SYMBOL_NAME(sys_init_module).long SYMBOL_NAME(sys_delete_module).long SYMBOL_NAME(sys_get_kernel_syms) /* 130 */.long SYMBOL_NAME(sys_quotactl).long SYMBOL_NAME(sys_getpgid).long SYMBOL_NAME(sys_fchdir).long SYMBOL_NAME(sys_bdflush).long SYMBOL_NAME(sys_sysfs) /* 135 */.long SYMBOL_NAME(sys_personality).long SYMBOL_NAME(sys_ni_syscall) /* for afs_syscall */.long SYMBOL_NAME(sys_setfsuid16).long SYMBOL_NAME(sys_setfsgid16).long SYMBOL_NAME(sys_llseek) /* 140 */.long SYMBOL_NAME(sys_getdents).long SYMBOL_NAME(sys_select).long SYMBOL_NAME(sys_flock).long SYMBOL_NAME(sys_msync).long SYMBOL_NAME(sys_readv) /* 145 */.long SYMBOL_NAME(sys_writev).long SYMBOL_NAME(sys_getsid).long SYMBOL_NAME(sys_fdatasync).long SYMBOL_NAME(sys_sysctl).long SYMBOL_NAME(sys_mlock) /* 150 */.long SYMBOL_NAME(sys_munlock).long SYMBOL_NAME(sys_mlockall).long SYMBOL_NAME(sys_munlockall).long SYMBOL_NAME(sys_sched_setparam).long SYMBOL_NAME(sys_sched_getparam) /* 155 */.long SYMBOL_NAME(sys_sched_setscheduler).long SYMBOL_NAME(sys_sched_getscheduler).long SYMBOL_NAME(sys_sched_yield).long SYMBOL_NAME(sys_sched_get_priority_max).long SYMBOL_NAME(sys_sched_get_priority_min) /* 160 */.long SYMBOL_NAME(sys_sched_rr_get_interval).long SYMBOL_NAME(sys_nanosleep).long SYMBOL_NAME(sys_mremap).long SYMBOL_NAME(sys_setresuid16).long SYMBOL_NAME(sys_getresuid16) /* 165 */.long SYMBOL_NAME(sys_vm86).long SYMBOL_NAME(sys_query_module).long SYMBOL_NAME(sys_poll).long SYMBOL_NAME(sys_nfsservctl).long SYMBOL_NAME(sys_setresgid16) /* 170 */.long SYMBOL_NAME(sys_getresgid16).long SYMBOL_NAME(sys_prctl).long SYMBOL_NAME(sys_rt_sigreturn).long SYMBOL_NAME(sys_rt_sigaction).long SYMBOL_NAME(sys_rt_sigprocmask) /* 175 */.long SYMBOL_NAME(sys_rt_sigpending).long SYMBOL_NAME(sys_rt_sigtimedwait).long SYMBOL_NAME(sys_rt_sigqueueinfo).long SYMBOL_NAME(sys_rt_sigsuspend).long SYMBOL_NAME(sys_pread) /* 180 */.long SYMBOL_NAME(sys_pwrite).long SYMBOL_NAME(sys_chown16).long SYMBOL_NAME(sys_getcwd).long SYMBOL_NAME(sys_capget).long SYMBOL_NAME(sys_capset) /* 185 */.long SYMBOL_NAME(sys_sigaltstack).long SYMBOL_NAME(sys_sendfile).long SYMBOL_NAME(sys_ni_syscall) /* streams1 */.long SYMBOL_NAME(sys_ni_syscall) /* streams2 */.long SYMBOL_NAME(sys_vfork) /* 190 */.long SYMBOL_NAME(sys_getrlimit).long SYMBOL_NAME(sys_mmap2).long SYMBOL_NAME(sys_truncate64).long SYMBOL_NAME(sys_ftruncate64).long SYMBOL_NAME(sys_stat64) /* 195 */.long SYMBOL_NAME(sys_lstat64).long SYMBOL_NAME(sys_fstat64).long SYMBOL_NAME(sys_lchown).long SYMBOL_NAME(sys_getuid).long SYMBOL_NAME(sys_getgid) /* 200 */.long SYMBOL_NAME(sys_geteuid).long SYMBOL_NAME(sys_getegid).long SYMBOL_NAME(sys_setreuid).long SYMBOL_NAME(sys_setregid).long SYMBOL_NAME(sys_getgroups) /* 205 */.long SYMBOL_NAME(sys_setgroups).long SYMBOL_NAME(sys_fchown).long SYMBOL_NAME(sys_setresuid).long SYMBOL_NAME(sys_getresuid).long SYMBOL_NAME(sys_setresgid) /* 210 */.long SYMBOL_NAME(sys_getresgid).long SYMBOL_NAME(sys_chown).long SYMBOL_NAME(sys_setuid).long SYMBOL_NAME(sys_setgid).long SYMBOL_NAME(sys_setfsuid) /* 215 */.long SYMBOL_NAME(sys_setfsgid).long SYMBOL_NAME(sys_pivot_root).long SYMBOL_NAME(sys_mincore).long SYMBOL_NAME(sys_madvise).long SYMBOL_NAME(sys_getdents64) /* 220 */.long SYMBOL_NAME(sys_fcntl64).long SYMBOL_NAME(sys_ni_syscall) /* reserved for TUX */.long SYMBOL_NAME(sys_ni_syscall) /* Reserved for Security */.long SYMBOL_NAME(sys_gettid).long SYMBOL_NAME(sys_readahead) /* 225 */IRQ Event
When an IRQ comes, the task that is running is interrupted in order to service the IRQ Handler.
After the IRQ is handled, control returns backs exactly to point of interrupt, like nothing happened.Running Task|-----------| (3)NORMAL | | | [break execution] IRQ HandlerEXECUTION (1)| | | ------------->|---------|| \|/ | | | does |IRQ (2)---->| .. |-----> | some || | |<----- | work |BACK TO | | | | | ..(4). |NORMAL (6)| \|/ | <-------------|_________|EXECUTION |___________| [return to code](5)USER MODE KERNEL MODEUser->Kernel Mode Transition caused by IRQ eventThe numbered steps below refer to the sequence of events in the diagram above:
- Process is executing
- IRQ comes while the task is running.
- Task is interrupted to call an "Interrupt handler".
- The "Interrupt handler" code is executed.
- Control returns back to task user mode (as if nothing happened)
- Process returns back to normal execution
Special interest has the Timer IRQ, coming every TIMER ms to manage:
- Alarms
- System and task counters (used by schedule to decide when stop a process or for accounting)
- Multitasking based on wake up mechanism after TIMESLICE time.
Multitasking
Mechanism
The key point of modern OSs is the "Task". The Task is an application running in memory sharing all resources (included CPU
and Memory) with other Tasks.
This "resource sharing" is managed by the "Multitasking Mechanism". The Multitasking Mechanism switches from one task to
another after a "timeslice" time. Users have the "illusion" that they own all resources. We can also imagine a single user scenario,
where a user can have the "illusion" of running many tasks at the same time.
To implement this multitasking, the task uses "the state" variable, which can be:
- READY, ready for execution
- BLOCKED, waiting for a resource
The task state is managed by its presence in a relative list: READY list and BLOCKED list.
Task Switching
The movement from one task to another is called ''Task Switching''. many computers have a hardware instruction which automatically
performs this operation. Task Switching occurs in the following cases:
- After Timeslice ends: we need to schedule a "Ready for execution" task and give it access.
- When a Task has to wait for a device: we need to schedule a new task and switch to it *
* We schedule another task to prevent "Busy Form Waiting", which occurs when we are waiting for a device instead performing
other work.Task Switching is managed by the "Schedule" entity.
Timer | |IRQ | | Schedule| | | ________________________|----->| Task 1 |<------------------>|(1)Chooses a Ready Task || | | |(2)Task Switching || |___________| |________________________|| | | /|\| | | || | | || | | || | | ||----->| Task 2 |<-------------------------------|| | | || |___________| |. . . . .. . . . .. . . . .| | | || | | |------>| Task N |<--------------------------------| ||___________|Task Switching based on TimeSliceA typical Timeslice for Linux is about 10 ms.
| || | Resource _____________________________| Task 1 |----------->|(1) Enqueue Resource request || | Access |(2) Mark Task as blocked || | |(3) Choose a Ready Task ||___________| |(4) Task Switching ||_____________________________|||| | || | || Task 2 |<-------------------------| || ||___________|Task Switching based on Waiting for a Resource
Microkernel vs Monolithic OS
Until now we viewed so called Monolithic OS, but there is also another kind of OS: ''Microkernel''.
A Microkernel OS uses Tasks, not only for user mode processes, but also as a real kernel manager, like Floppy-Task,
HDD-Task, Net-Task and so on. Some examples are Amoeba, and Mach.PROs and CONTROs of Microkernel OS
PROS:
- OS is simpler to maintain because each Task manages a single kind of operation. So if you want to modify networking,
you modify Net-Task (ideally, if it is not needed a structural update).CONS:
- Performances are worse than Monolithic OS, because you have to add 2*TASK_SWITCH times (the first to enter the
specific Task, the second to go out from it).Microkernels are a good didactic example (like Minix) but they are not ''optimal'', so not really suitable. Linux uses a few Tasks,
called "Kernel Threads" to implement a little microkernel structure (like kswapd, which is used to retrieve memory pages from
mass storage). In this case there are no problems with perfomance because swapping is a very slow job.Networking
ISO OSI levels
Standard ISO-OSI describes a network architecture with the following levels:
- Physical level (examples: PPP and Ethernet)
- Data-link level (examples: PPP and Ethernet)
- Network level (examples: IP, and X.25)
- Transport level (examples: TCP, UDP)
- Session level (SSL)
- Presentation level (FTP binary-ascii coding)
- Application level (applications like Netscape)
The first 2 levels listed above are often implemented in hardware. Next levels are in software (or firmware for routers).
Many protocols are used by an OS: one of these is TCP/IP (the most important living on 3-4 levels).
What does the kernel?
The kernel doesn't know anything (only addresses) about first 2 levels of ISO-OSI.
In RX it:
- Manages handshake with low levels devices (like ethernet card or modem) receiving "frames" from them.
- Builds TCP/IP "packets" from "frames" (like Ethernet or PPP ones),
- Convers ''packets'' in ''sockets'' passing them to the right application (using port number) or
- Forwards packets to the right queue
frames packets socketsNIC ---------> Kernel ----------> Application| packets--------------> Forward- RX -In TX stage it:
- Converts sockets or
- Queues datas into TCP/IP ''packets''
- Splits ''packets" into "frames" (like Ethernet or PPP ones)
- Sends ''frames'' using HW drivers
sockets packets framesApplication ---------> Kernel ----------> NICpackets /|\Forward -------------------- TX -Virtual Memory
Segmentation
Segmentation is the first method to solve memory allocation problems: it allows you to compile source code without caring
where the application will be placed in memory. As a matter of fact, this feature helps applications developers to develop
in a independent fashion from the OS e also from the hardware.| Stack || | || \|/ || Free || /|\ | Segment <---> Process| | || Heap || Data uninitialized || Data initialized || Code ||____________________|Segment
We can say that a segment is the logical entity of an application, or the image of the application in memory.
When programming, we don't care where our data is put in memory, we only care about the offset inside our
segment (our application).
We use to assign a Segment to each Process and vice versa. In Linux this is not true. Linux uses only 4
segments for either Kernel and all Processes.
Problems of Segmentation
____________________----->| |----->| IN | Segment A | OUT____________________ | |____________________|| |____| | || Segment B | | Segment B || |____ | ||____________________| | |____________________|| | Segment C || |____________________|----->| Segment D |----->IN |____________________| OUTSegmentation problemIn the diagram above, we want to get exit processes A, and D and enter process B. As we can see there is enough space for B,
but we cannot split it in 2 pieces, so we CANNOT load it (memory out).
The reason this problem occurs is because pure segments are continuous areas (because they are logical areas) and cannot
be split.Pagination
____________________| Page 1 ||____________________|| Page 2 ||____________________|| .. | Segment <---> Process|____________________|| Page n ||____________________|| ||____________________|| ||____________________|SegmentPagination splits memory in "n" pieces, each one with a fixed length.
A process may be loaded in one or more Pages. When memory is freed, all pages are freed (see Segmentation Problem, before).
Pagination is also used for another important purpose, "Swapping". If a page is not present in physical memory then it generates
an EXCEPTION, that will make the Kernel search for a new page in storage memory. This mechanism allow OS to load more
applications than the ones allowed by physical memory only.Pagination Problem
____________________Page X | Process Y ||____________________|| || WASTE || SPACE ||____________________|Pagination ProblemIn the diagram above, we can see what is wrong with the pagination policy: when a Process Y loads into Page X, ALL memory
space of the Page is allocated, so the remaining space at the end of Page is wasted.Segmentation and Pagination
How can we solve segmentation and pagination problems? Using either 2 policies.
| .. ||____________________|----->| Page 1 || |____________________|| | .. |____________________ | |____________________|| | |---->| Page 2 || Segment X | ----| |____________________|| | | | .. ||____________________| | |____________________|| | .. || |____________________||---->| Page 3 ||____________________|| .. |Process X, identified by Segment X, is split in 3 pieces and each of one is loaded in a page.
We do not have:
- Segmentation problem: we allocate per Pages, so we also free Pages and we manage free space in an optimized way.
- Pagination problem: only last page wastes space, but we can decide to use very small pages, for example 4096 bytes
length (losing at maximum 4096*N_Tasks bytes) and manage hierarchical paging (using 2 or 3 levels of paging)| | | || | Offset2 | Value || | /|\| |Offset1 | |----- | | |/|\ | | | | | || | | | \|/| || | | ------>| |\|/ | | | |Base Paging Address ---->| | | || ....... | | ....... || | | |Hierarchical Paging
Linux Startup
We start the Linux kernel first from C code executed from ''startup_32:'' asm label:
|startup_32:|start_kernel|lock_kernel|trap_init|init_IRQ|sched_init|softirq_init|time_init|console_init|#ifdef CONFIG_MODULES|init_modules|#endif|kmem_cache_init|sti|calibrate_delay|mem_init|kmem_cache_sizes_init|pgtable_cache_init|fork_init|proc_caches_init|vfs_caches_init|buffer_init|page_cache_init|signals_init|#ifdef CONFIG_PROC_FS|proc_root_init|#endif|#if defined(CONFIG_SYSVIPC)|ipc_init|#endif|check_bugs|smp_init|rest_init|kernel_thread|unlock_kernel|cpu_idle
- startup_32 [arch/i386/kernel/head.S]
- start_kernel [init/main.c]
- lock_kernel [include/asm/smplock.h]
- trap_init [arch/i386/kernel/traps.c]
- init_IRQ [arch/i386/kernel/i8259.c]
- sched_init [kernel/sched.c]
- softirq_init [kernel/softirq.c]
- time_init [arch/i386/kernel/time.c]
- console_init [drivers/char/tty_io.c]
- init_modules [kernel/module.c]
- kmem_cache_init [mm/slab.c]
- sti [include/asm/system.h]
- calibrate_delay [init/main.c]
- mem_init [arch/i386/mm/init.c]
- kmem_cache_sizes_init [mm/slab.c]
- pgtable_cache_init [arch/i386/mm/init.c]
- fork_init [kernel/fork.c]
- proc_caches_init
- vfs_caches_init [fs/dcache.c]
- buffer_init [fs/buffer.c]
- page_cache_init [mm/filemap.c]
- signals_init [kernel/signal.c]
- proc_root_init [fs/proc/root.c]
- ipc_init [ipc/util.c]
- check_bugs [include/asm/bugs.h]
- smp_init [init/main.c]
- rest_init
- kernel_thread [arch/i386/kernel/process.c]
- unlock_kernel [include/asm/smplock.h]
- cpu_idle [arch/i386/kernel/process.c]
The last function ''rest_init'' does the following:
- launches the kernel thread ''init''
- calls unlock_kernel
- makes the kernel run cpu_idle routine, that will be the idle loop executing when nothing is scheduled
In fact the start_kernel procedure never ends. It will execute cpu_idle routine endlessly.
Follows ''init'' description, which is the first Kernel Thread:|init|lock_kernel|do_basic_setup|mtrr_init|sysctl_init|pci_init|sock_init|start_context_thread|do_init_calls|(*call())-> kswapd_init|prepare_namespace|free_initmem|unlock_kernel|execve
Kernel Modules
Linux Kernel modules are pieces of code (examples: fs, net, and hw driver) running in kernel mode that you can add at runtime.
The Linux core cannot be modularized: scheduling and interrupt management or core network, and so on.
Under "/lib/modules/KERNEL_VERSION/" you can find all the modules installed on your system.Module loading and unloading
To load a module, type the following:
insmod MODULE_NAME parametersexample: insmod ne io=0x300 irq=9NOTE: You can use modprobe in place of insmod if you want the kernel automatically search some parameter
(for example when using PCI driver, or if you have specified parameter under /etc/conf.modules file).
To unload a module, type the following:rmmod MODULE_NAMEModule definition
A module always contains:
- "init_module" function, executed at insmod (or modprobe) command
- "cleanup_module" function, executed at rmmod command
If these functions are not in the module, you need to add 2 macros to specify what functions will act as init and exit module:
- module_init(FUNCTION_NAME)
- module_exit(FUNCTION_NAME)
NOTE: a module can "see" a kernel variable only if it has been exported (with macro EXPORT_SYMBOL).
A useful trick for adding flexibility to the kernel
// kernel sources sidevoid (*foo_function_pointer)(void *);if (foo_function_pointer)(foo_function_pointer)(parameter);// module sideextern void (*foo_function_pointer)(void *);void my_function(void *parameter) {//My code}int init_module() {foo_function_pointer = &my_function;}int cleanup_module() {foo_function_pointer = NULL;}This simple trick allows you to have very high flexibility in your Kernel, because only when you load the module you'll make
"my_function" routine execute. This routine will do everything you want to do: for example ''rshaper'' module, which controls
bandwidth input traffic from the network, works in this kind of matter.
Notice that the whole module mechanism is possible thanks to some global variables exported to modules, such as head list
(allowing you to extend the list as much as you want). Typical examples are fs, generic devices (char, block, net, telephony).
You have to prepare the kernel to accept your new module; in some cases you have to create an infrastructure (like telephony
one, that was recently created) to be as standard as possible.Proc directory
Proc fs is located in the /proc directory, which is a special directory allowing you to talk directly with kernel.
Linux uses ''proc'' directory to support direct kernel communications: this is necessary in many cases, for
example when you want see main processes data structures or enable ''proxy-arp'' feature on one interface and
not in others, you want to change max number of threads, or if you want to debug some bus state, like ISA or PCI,
to know what cards are installed and what I/O addresses and IRQs are assigned to them.|-- bus| |-- pci| | |-- 00| | | |-- 00.0| | | |-- 01.0| | | |-- 07.0| | | |-- 07.1| | | |-- 07.2| | | |-- 07.3| | | |-- 07.4| | | |-- 07.5| | | |-- 09.0| | | |-- 0a.0| | | `-- 0f.0| | |-- 01| | | `-- 00.0| | `-- devices| `-- usb|-- cmdline|-- cpuinfo|-- devices|-- dma|-- dri| `-- 0| |-- bufs| |-- clients| |-- mem| |-- name| |-- queues| |-- vm| `-- vma|-- driver|-- execdomains|-- filesystems|-- fs|-- ide| |-- drivers| |-- hda -> ide0/hda| |-- hdc -> ide1/hdc| |-- ide0| | |-- channel| | |-- config| | |-- hda| | | |-- cache| | | |-- capacity| | | |-- driver| | | |-- geometry| | | |-- identify| | | |-- media| | | |-- model| | | |-- settings| | | |-- smart_thresholds| | | `-- smart_values| | |-- mate| | `-- model| |-- ide1| | |-- channel| | |-- config| | |-- hdc| | | |-- capacity| | | |-- driver| | | |-- identify| | | |-- media| | | |-- model| | | `-- settings| | |-- mate| | `-- model| `-- via|-- interrupts|-- iomem|-- ioports|-- irq| |-- 0| |-- 1| |-- 10| |-- 11| |-- 12| |-- 13| |-- 14| |-- 15| |-- 2| |-- 3| |-- 4| |-- 5| |-- 6| |-- 7| |-- 8| |-- 9| `-- prof_cpu_mask|-- kcore|-- kmsg|-- ksyms|-- loadavg|-- locks|-- meminfo|-- misc|-- modules|-- mounts|-- mtrr|-- net| |-- arp| |-- dev| |-- dev_mcast| |-- ip_fwchains| |-- ip_fwnames| |-- ip_masquerade| |-- netlink| |-- netstat| |-- packet| |-- psched| |-- raw| |-- route| |-- rt_acct| |-- rt_cache| |-- rt_cache_stat| |-- snmp| |-- sockstat| |-- softnet_stat| |-- tcp| |-- udp| |-- unix| `-- wireless|-- partitions|-- pci|-- scsi| |-- ide-scsi| | `-- 0| `-- scsi|-- self -> 2069|-- slabinfo|-- stat|-- swaps|-- sys| |-- abi| | |-- defhandler_coff| | |-- defhandler_elf| | |-- defhandler_lcall7| | |-- defhandler_libcso| | |-- fake_utsname| | `-- trace| |-- debug| |-- dev| | |-- cdrom| | | |-- autoclose| | | |-- autoeject| | | |-- check_media| | | |-- debug| | | |-- info| | | `-- lock| | `-- parport| | |-- default| | | |-- spintime| | | `-- timeslice| | `-- parport0| | |-- autoprobe| | |-- autoprobe0| | |-- autoprobe1| | |-- autoprobe2| | |-- autoprobe3| | |-- base-addr| | |-- devices| | | |-- active| | | `-- lp| | | `-- timeslice| | |-- dma| | |-- irq| | |-- modes| | `-- spintime| |-- fs| | |-- binfmt_misc| | |-- dentry-state| | |-- dir-notify-enable| | |-- dquot-nr| | |-- file-max| | |-- file-nr| | |-- inode-nr| | |-- inode-state| | |-- jbd-debug| | |-- lease-break-time| | |-- leases-enable| | |-- overflowgid| | `-- overflowuid| |-- kernel| | |-- acct| | |-- cad_pid| | |-- cap-bound| | |-- core_uses_pid| | |-- ctrl-alt-del| | |-- domainname| | |-- hostname| | |-- modprobe| | |-- msgmax| | |-- msgmnb| | |-- msgmni| | |-- osrelease| | |-- ostype| | |-- overflowgid| | |-- overflowuid| | |-- panic| | |-- printk| | |-- random| | | |-- boot_id| | | |-- entropy_avail| | | |-- poolsize| | | |-- read_wakeup_threshold| | | |-- uuid| | | `-- write_wakeup_threshold| | |-- rtsig-max| | |-- rtsig-nr| | |-- sem| | |-- shmall| | |-- shmmax| | |-- shmmni| | |-- sysrq| | |-- tainted| | |-- threads-max| | `-- version| |-- net| | |-- 802| | |-- core| | | |-- hot_list_length| | | |-- lo_cong| | | |-- message_burst| | | |-- message_cost| | | |-- mod_cong| | | |-- netdev_max_backlog| | | |-- no_cong| | | |-- no_cong_thresh| | | |-- optmem_max| | | |-- rmem_default| | | |-- rmem_max| | | |-- wmem_default| | | `-- wmem_max| | |-- ethernet| | |-- ipv4| | | |-- conf| | | | |-- all| | | | | |-- accept_redirects| | | | | |-- accept_source_route| | | | | |-- arp_filter| | | | | |-- bootp_relay| | | | | |-- forwarding| | | | | |-- log_martians| | | | | |-- mc_forwarding| | | | | |-- proxy_arp| | | | | |-- rp_filter| | | | | |-- secure_redirects| | | | | |-- send_redirects| | | | | |-- shared_media| | | | | `-- tag| | | | |-- default| | | | | |-- accept_redirects| | | | | |-- accept_source_route| | | | | |-- arp_filter| | | | | |-- bootp_relay| | | | | |-- forwarding| | | | | |-- log_martians| | | | | |-- mc_forwarding| | | | | |-- proxy_arp| | | | | |-- rp_filter| | | | | |-- secure_redirects| | | | | |-- send_redirects| | | | | |-- shared_media| | | | | `-- tag| | | | |-- eth0| | | | | |-- accept_redirects| | | | | |-- accept_source_route| | | | | |-- arp_filter| | | | | |-- bootp_relay| | | | | |-- forwarding| | | | | |-- log_martians| | | | | |-- mc_forwarding| | | | | |-- proxy_arp| | | | | |-- rp_filter| | | | | |-- secure_redirects| | | | | |-- send_redirects| | | | | |-- shared_media| | | | | `-- tag| | | | |-- eth1| | | | | |-- accept_redirects| | | | | |-- accept_source_route| | | | | |-- arp_filter| | | | | |-- bootp_relay| | | | | |-- forwarding| | | | | |-- log_martians| | | | | |-- mc_forwarding| | | | | |-- proxy_arp| | | | | |-- rp_filter| | | | | |-- secure_redirects| | | | | |-- send_redirects| | | | | |-- shared_media| | | | | `-- tag| | | | `-- lo| | | | |-- accept_redirects| | | | |-- accept_source_route| | | | |-- arp_filter| | | | |-- bootp_relay| | | | |-- forwarding| | | | |-- log_martians| | | | |-- mc_forwarding| | | | |-- proxy_arp| | | | |-- rp_filter| | | | |-- secure_redirects| | | | |-- send_redirects| | | | |-- shared_media| | | | `-- tag| | | |-- icmp_echo_ignore_all| | | |-- icmp_echo_ignore_broadcasts| | | |-- icmp_ignore_bogus_error_responses| | | |-- icmp_ratelimit| | | |-- icmp_ratemask| | | |-- inet_peer_gc_maxtime| | | |-- inet_peer_gc_mintime| | | |-- inet_peer_maxttl| | | |-- inet_peer_minttl| | | |-- inet_peer_threshold| | | |-- ip_autoconfig| | | |-- ip_conntrack_max| | | |-- ip_default_ttl| | | |-- ip_dynaddr| | | |-- ip_forward| | | |-- ip_local_port_range| | | |-- ip_no_pmtu_disc| | | |-- ip_nonlocal_bind| | | |-- ipfrag_high_thresh| | | |-- ipfrag_low_thresh| | | |-- ipfrag_time| | | |-- neigh| | | | |-- default| | | | | |-- anycast_delay| | | | | |-- app_solicit| | | | | |-- base_reachable_time| | | | | |-- delay_first_probe_time| | | | | |-- gc_interval| | | | | |-- gc_stale_time| | | | | |-- gc_thresh1| | | | | |-- gc_thresh2| | | | | |-- gc_thresh3| | | | | |-- locktime| | | | | |-- mcast_solicit| | | | | |-- proxy_delay| | | | | |-- proxy_qlen| | | | | |-- retrans_time| | | | | |-- ucast_solicit| | | | | `-- unres_qlen| | | | |-- eth0| | | | | |-- anycast_delay| | | | | |-- app_solicit| | | | | |-- base_reachable_time| | | | | |-- delay_first_probe_time| | | | | |-- gc_stale_time| | | | | |-- locktime| | | | | |-- mcast_solicit| | | | | |-- proxy_delay| | | | | |-- proxy_qlen| | | | | |-- retrans_time| | | | | |-- ucast_solicit| | | | | `-- unres_qlen| | | | |-- eth1| | | | | |-- anycast_delay| | | | | |-- app_solicit| | | | | |-- base_reachable_time| | | | | |-- delay_first_probe_time| | | | | |-- gc_stale_time| | | | | |-- locktime| | | | | |-- mcast_solicit| | | | | |-- proxy_delay| | | | | |-- proxy_qlen| | | | | |-- retrans_time| | | | | |-- ucast_solicit| | | | | `-- unres_qlen| | | | `-- lo| | | | |-- anycast_delay| | | | |-- app_solicit| | | | |-- base_reachable_time| | | | |-- delay_first_probe_time| | | | |-- gc_stale_time| | | | |-- locktime| | | | |-- mcast_solicit| | | | |-- proxy_delay| | | | |-- proxy_qlen| | | | |-- retrans_time| | | | |-- ucast_solicit| | | | `-- unres_qlen| | | |-- route| | | | |-- error_burst| | | | |-- error_cost| | | | |-- flush| | | | |-- gc_elasticity| | | | |-- gc_interval| | | | |-- gc_min_interval| | | | |-- gc_thresh| | | | |-- gc_timeout| | | | |-- max_delay| | | | |-- max_size| | | | |-- min_adv_mss| | | | |-- min_delay| | | | |-- min_pmtu| | | | |-- mtu_expires| | | | |-- redirect_load| | | | |-- redirect_number| | | | `-- redirect_silence| | | |-- tcp_abort_on_overflow| | | |-- tcp_adv_win_scale| | | |-- tcp_app_win| | | |-- tcp_dsack| | | |-- tcp_ecn| | | |-- tcp_fack| | | |-- tcp_fin_timeout| | | |-- tcp_keepalive_intvl| | | |-- tcp_keepalive_probes| | | |-- tcp_keepalive_time| | | |-- tcp_max_orphans| | | |-- tcp_max_syn_backlog| | | |-- tcp_max_tw_buckets| | | |-- tcp_mem| | | |-- tcp_orphan_retries| | | |-- tcp_reordering| | | |-- tcp_retrans_collapse| | | |-- tcp_retries1| | | |-- tcp_retries2| | | |-- tcp_rfc1337| | | |-- tcp_rmem| | | |-- tcp_sack| | | |-- tcp_stdurg| | | |-- tcp_syn_retries| | | |-- tcp_synack_retries| | | |-- tcp_syncookies| | | |-- tcp_timestamps| | | |-- tcp_tw_recycle| | | |-- tcp_window_scaling| | | `-- tcp_wmem| | `-- unix| | `-- max_dgram_qlen| |-- proc| `-- vm| |-- bdflush| |-- kswapd| |-- max-readahead| |-- min-readahead| |-- overcommit_memory| |-- page-cluster| `-- pagetable_cache|-- sysvipc| |-- msg| |-- sem| `-- shm|-- tty| |-- driver| | `-- serial| |-- drivers| |-- ldisc| `-- ldiscs|-- uptime`-- versionIn the directory there are also all the tasks using PID as file names (you have access to all Task information, like path of
binary file, memory used, and so on).
Starting CyN
Since CyN is based on HURD and menuetOS linux systems we used the same schemas but changing the kernel operating mode.
CyN kernel works with this specific schema:| |------->| System Call x | (devices)| | | | [sys_read()] || ... | | | || network_call(x)|-------- | || [_node(xi)] * * *| ... | |Neuronal_ || system_proc(x) |-------- |Map(xi) *| [net_pid(xi)]* | * *| ... | ------->| System Call x | (data structures)| | | [sys_getpid()]|| |NODE MODE KERNEL MODEKernel can simulate the neuron structure only if the basis schema is taked:
Why develop an Operating System on CyN and not take advantage of some others?
Answer is simple, the fundamental purpose of an operating system (OS) is to enable a variety of programs to share a single
computer efficiently and productively. This demands memory protection, preemptively scheduled timesharing, coordinated
access to I/O peripherals, and other services. In addition, an OS can allow several users to share a computer. In this case,
efficiency demands services that protect users from harming each other, enable them to share without prior arrangement,
and mediate access to physical devices. But and Network design implies some adding protocol development and other important
consideration was detailed in this paper CyN and HURD have some differences between Operative Network against Imperative
Network
.
On today's computer systems, programmers usually implement these goals through a large program called the kernel. Since this
program must be accessible to all user programs, it is the natural place to add functionality to the system. Since the only model
for process interaction is that of specific, individual services provided by the kernel, no one creates other places to add
functionality. As time goes by, more and more is added to the kernel.
A traditional system allows users to add components to a kernel only if they both understand most of it and have a privileged
status within the system. Testing new components requires a much more painful edit-compile-debug cycle than testing other
programs. It cannot be done while others are using the system. Bugs usually cause fatal system crashes, further disrupting
others' use of the system. The entire kernel is usually non-pageable. (There are systems with pageable kernels, but deciding
what can be paged is difficult and error prone. Usually the mechanisms are complex, making them difficult to use even when
adding simple extensions.)
Because of these restrictions, functionality which properly belongs behind the wall of a traditional kernel is usually left out of
systems unless it is absolutely mandatory. Many good ideas, best done with an open/read/write interface cannot be implemented
because of the problems inherent in the monolithic nature of a traditional system. Further, even among those with the endurance
to implement new ideas, only those who are privileged users of their computers can do so. The software copyright system darkens
the mire by preventing unlicensed people from even reading the kernel source.
Some systems have tried to address these difficulties. Smalltalk and the Lisp Machine both represented one method of getting
around the problem. System code is not distinguished from user code; all of the system is accessible to the user and can be
changed as need be. Both systems were built around languages that facilitated such easy replacement and extension, and were
moderately successful. But they both were fairly poor at insulating users and programs from each other, failing one of the principal
goals of OS design.
Most projects that use the microkernel carry on the hard-to-change tradition of OS design. The internal structure is different, but
the same heavy barrier between user and system remains. The single-servers, while fairly easy to construct, inherit all the
deficiencies of the monolithic kernels.
A multi-server divides the kernel functionality up into logical blocks with well-defined interfaces. Properly done, it is easier to
make changes and add functionality. So most multi-server projects do somewhat better. Much more of the system is pageable.
You can debug the system more easily. You can test new system components without interfering with other users. But the wall
between user and system remains; no user can cross it without special privilege.
By contrast, CyN is designed to make the area of system code as limited as possible. Programs are required to communicate only
with a few essential parts of the kernel; the rest of the system is replaceable dynamically. Users can use whatever parts of
the remainder of the system they want, and can easily add components themselves for other users to take advantage of.
No mutual trust need exist in advance for users to use each other's services, nor does the system become vulnerable by
trusting the services of arbitrary users.
This has been done by identifying those system components which users must use in order to communicate with each other.
One of these is responsible for identifying users' identities and is called the authentication server. In order to establish each
other's identities, programs must communicate, each with an authentication server they trust. Another component establishes
control over system components by the superuser, provides global bookkeeping operations, and is called the process server.
Not all user programs need to communicate with the process server; it is only necessary for programs which require its services.
Likewise, the authentication server is only necessary for programs that wish to communicate their identity to another like a
develop researched and implemented on Bosne and Herzegovine Union at 2002 where the sistem calls and requested information
needs a strict authentication against hardware design. None of the remaining services carry any special status; not the network
implementation, the filesystems, the program execution mechanism (including setuid), or any others.
What this design provides is completely novel to the Unix world. Until now, OSs have kept huge portions of their functionality
in the realm of system code, thus preventing its modification and extension except in extreme need. Users cannot replace parts
of the system in their programs no matter how much easier that would make their task, and system managers are loath to install
random tweaks off the net into their kernels.
In the CyN, users can change almost all of the things that are decided for them in advance by traditional systems. In combination
with the network descriptors each node operator can design the work around operation, made the “node kernel mirror” work
schema as their own needs.
It permit to all network interact with other networks independiently of the process to be requested or working. This idea is
simple: If the operator A decide work with high cryptographical scheme he decide how the other nodes processors work with
that scheme, so if the operator of B node decide work with graphics the entire network process only per B request the processor
needs.Problems And Walk Around CyN Network Bugs
An I/O server will provide the terminal semantics of Posix. The GNU C Library has features for keeping track of the controlling
terminal and for arranging to have proper job control signals sent at the proper times, as well as features for obeying keyboard
and hangup signals.
Programs will be able to insert a terminal driver into communications channels in a variety of ways. Servers like rlogind will be
able to insert the terminal protocol onto their network communication port. Pseudo-terminals will not be necessary, though they
will be provided for backward compatibility with older programs. No programs in GNU will depend on them.
Nothing about a terminal driver is forced upon users. A terminal driver allows a user to get at the underlying communications
channel easily, to bypass itself on an as-needed basis or altogether, or to substitute a different terminal driver-like program.
In the last case, provided the alternate program implements the necessary interfaces, it will be used by the C Library exactly
as if it were the ordinary terminal driver.
Because of this flexibility, the original terminal driver will not provide complex line editing features, restricting itself to the
behavior found in Posix and BSD. In time, there will be a readline-based terminal driver, which will provide complex line-editing
features for those users who want them.
The terminal driver will probably not provide good support for the high-volume, rapid data transmission required by UUCP or SLIP.
Those programs do not need any of its features. Instead they will be use the underlying spector device ports for terminals,
which support moving large amounts of data efficiently or process by operators choose.4. Kernel Basis (HURD and menuetOS based)
The Hurd uses Mach ports primarily as methods for communicating between users and servers. (A Mach port is a communication
point on a Mach task where messages are sent and received.) Each port implements a particular set of protocols, representing
operations that can be undertaken on the underlying object represented by the port. Some of the protocols specified by the Hurd
are the I/O protocol, used for generic I/O operations; the file protocol, used for filesystem operations; the socket protocol, used
for network operations; and the process protocol, used for manipulating processes et al.
Most servers are accessed by opening files. Normally, when you open a file, you create a port associated with that file that is
owned by the server that owns the directory containing the file. For example, a disk-based filesystem will normally serve a large
number of ports, each of which represents an open file or directory. When a file is opened, the server creates a new port,
associates it with the file, and returns the port to the calling program.
However, a file can have a translator associated with it. In this case, rather than return its own port which refers to the contents
of the file, the server executes a translator program associated with that file. This translator is given a port to the actual contents
of the file, and is then asked to return a port to the original user to complete the open operation.
This mechanism is used for mount by having a translator associated with each mount point. When a program opens the mount
point, the translator (in this case, a program which understands the disk format of the mounted filesystem) is executed and
returns a port to the program. After the translator is started, it need not be run again unless it dies; the parent filesystem
retains a port to the translator to use in further requests.
The owner of a file can associate a translator with it without special permission. This means that any program can be specified
as a translator. Obviously the system will not work properly if the translator does not implement the file protocol correctly.
However, the Hurd is constructed so that the worst possible consequence is an interruptible hang.
One way to use translators is to access hierarchically structured data using the file protocol. For example, all the complexity of
the user interface to the ftp program is removed. Users need only know that a particular directory represents FTP and can use
all the standard file manipulation commands (e.g ls or cp) to access the remote system, rather than learning a new set. Similarly,
a simple translator could ease the complexity of tar or gzip. (Such transparent access would have some added cost, but it would
be convenient.)
MenuetOS is a hobby Operating System for the PC written entirely in 64bit assembly language, and released under License. It
supports 64 and 32 bit x86 assembly programming for smaller, faster and less resource hungry applications.Menuet has no roots within unix or the posix standards, nor is it based on any particular operating system. The design goal has
been to remove the extra layers between different parts of an OS, which normally complicates programming and create bugs.Menuet's application structure is not specifically reserved for asm programming since the header can be produced with practically
any other language. However, the overall application programming design is intended for easy 64/32 bit asm programming.
Menuets responsive GUI is easy to handle with assembly language.5. Clusterin Analisys
Clustering usually brings to mind dozens of servers working together to crack the latest unbreakable encryption, or to break
the world's record for calculating the most significant digits for pi. Although clustering plays a very important role in academia
and research, it can also be important in enterprise computing, and especially in enterprise network computing, where guaranteed
bandwidth does not guarantee the quickest responses for network applications.Network clustering shares a load among servers in a predetermined method. The load distribution is unknown to the user's
application and, as far as the user is concerned, he is communicating with one server. With the cluster, however, the user
will be sharing the server with fewer users, therefore improving the user's experience with the site. This technology has
become very popular for Web sites, where Web developers and marketing departments try to push the bottleneck of the
Web experience as close to the user as possible while also avoiding congestion.In the simplest form, a network cluster is a load balancer, as shown in the Figure " Load Balancer," which directs each packet
in a round-robin fashion to each of the servers in the cluster. This type of clustering is easy to implement and is easy to support.
However, a load balancer relies on lower level protocols for state information. With protocols like http,where a user session is
actually multiple http requests, such a load balancer cannot support dynamic content. Additionally, the secure socket layer,
which is used to encrypt traffic (such as credit card transactions), requires a key exchange with the destination that cannot be
shared with other servers.To support this requirement, the load balancer needs to maintain a state table to ensure that specific users' sessions are
always sent to the same server. This support is often called "sticky" session support by cluster software developers since each
session sticks to the same server. (Support of sticky sessions brings its own design considerations, such as further skewing of
round robin's less than ideal balancing.)Additionally, a simple load balancer does not provide the resiliency expected of a cluster solution. Although an algorithm
(protocol) can be supported to determine when one of the servers fails, the load balancer itself is a single point of failure
within the cluster.Many current network clustering solutions are based upon Ethernet transport. Ethernet has proven itself as a LAN protocol and
most enterprises have in house know-how and extensive experience with the technology.Early clustering and some proprietary clustering solutions require a proprietary physical connection between the load balancer
and the servers. The typical advantage of a proprietary connection is a quick recovery time in the case of failure; however, this
is often overkill in intranet and Internet network solutions, except in those cases where systems are only allowed minutes per
year of down time. Systems with proprietary physical connections also often use proprietary protocols that are closely
integrated with their choice of physical networking.The load balancer must not connect with the servers in the cluster; it can also be a logical system as opposed to a physical
system. For example, all the nodes in the cluster may themselves act as the balancer, using proprietary protocols or creative
designs such as virtual layer-2 or layer-3 addresses using open protocols.Network clusters, unlike their larger application clusters, often do not have support for additional distributed system support,
such as a distributed database. This often means database support, when needed, must be designed specifically for the solution.
For example, updates to a database could be made to a central server via a network connection, or a local database could be
updated with a batch process to replicate changes to take place at a scheduled time.To ensure reliability, the nodes in the cluster must communicate their statuses to each other. The method used can either
be a simple ping-like command to ensure network connectivity (a very weak method), or it can actually connect with the
servers to ensure the particular application is answering. Finally, the load balancer could remotely connect to the nodes
much like an administrator does, to ensure certain processes are running.Nodes participating in a cluster (which all production systems should have) will have a means of restarting processes and
notifying the administrator whenever there is a failure. Unlike a normal production server, multiple failures will ideally have
the node removed from the cluster since a "flapping" can cause additional interrupts to the user as the user's host-initiated
TCP stack tries to compensate with retransmissions due to lack of acknowledgements. This can seem like a slow responding
server to the user, whereas an earlier failover is less likely not to be recognized by the user.
6. Sphread Information Analisis Against Server Farm PlatformA server farm is a collection of computer servers usually maintained by an enterprise to accomplish server needs far beyond
the capability of one machine. Often, server farms will have both a primary and a backup server allocated to a single task,
so that in the event of the failure of the primary server, a backup server will take over the primary server's function.Server farms are typically co-located with the network switches and/or routers which enable communication between the
different parts of the cluster and the users of the cluster.Server farms are commonly used for cluster computing. Many modern supercomputers consist of giant server farms of
high-speed processors connected by either Gigabit Ethernet or custom interconnects such as Myrinet.Another common use of server farms is for web hosting.
Server farms are increasingly being used instead of or in addition to mainframe computers by large enterprises, although
server farms do not as yet reach the same reliability levels as mainframes. Because of the sheer number of computers in
large server farms, the failure of individual machines is a commonplace event, and the management of large server farms
needs to take this into account, by providing support for redundancy, automatic failover, and rapid reconfiguration of the
server cluster.The performance of the very largest server farms (thousands of processors and up) is typically limited by the performance
of the data center's cooling systems rather than by the performance of the processors. For this reason, the critical design
parameter for such systems tends to be performance per watt of generated heat, rather than performance per processor.By the way a server farm array technology is redundant and dependient by the switching and routing configuration also,
is the best way for manage a non demand data storage and linked NAT web servers, usually the most enterprises around
the world choose these technology, maybe choose a “node” for cluster on the internet network for public or private servers
(government and military).
Server farm only respond to the clients demand and it’s eficiently because the low process (data transactions usually) for
data and not processor mechanism for the clients usually taken.
7. Security Analisys Against Honeypots Research
A honeypot is a closely monitored network decoy serving several purposes: it can distract adversaries from more valuable
machines on a network, provide early warning about new attack and exploitation trends, or allow in-depth examination
of adversaries during and after exploitation of a honeypot. Deploying a physical honeypot is often time intensive and
expensive as different operating systems require specialized hardware and every honeypot requires its own physical system.
The value of a honeypot is determined by the information that we can obtain from it. Monitoring the data that enters and
leaves a honeypot lets us gather information that is not available to NIDS. For example, we can log the key strokes of an
interactive session even if encryption is used to protect the network traffic. To detect malicious behavior, NIDS require
signatures of known attacks and often fail to detect compromises that were unknown at the time it was deployed.
The paper titled "Insertion, Evasion and Denial Of Service: Eluding NIDS" by Thomas.H.Ptacek and Tim Newsham triggered
research in the field of eluding NIDS. Ever since this paper was published, people have been busy finding new techniques
of eluding NIDS. Most intrusion detection systems (IDS) generally have support for TCP-reassembly and the capability to
monitor sessions. Some of the DoS attacks focus on overflowing the stream-buffer cache of the IDS so that the stream
being monitored gets disrupted. An Insertion Attack sends packets to an end-system (victim) that will reject, but that the
IDS will think are valid, thus giving different streams to the IDS and target hosts. In comparison, an evasion attack sends
packets which the IDS rejects but the target host accepts, again giving different streams to the IDS and target. In order
to achieve these attacks, attackers also use packet fragmentation where the attack stream is broken into smaller ones.
We will now describe some of these evasion techniques.
Attack scenario:
Suppose the IDS fragmentation reassembly timeout is 15 seconds and the system is monitoring some Linux hosts which
have a default fragmentation reassembly timeout of 30 seconds. As shown below in Figure 1, after sending the first
fragment the attacker can send the second fragment with a delay of 15 seconds but still within 30 seconds.
Now, the victim reassembles the fragments whereas at the IDS the fragmentation reassembly timeout parameter kicks in and a
timeout occurs. Remember, here the second fragment received by the IDS will be dropped as the IDS has already lost the first
fragment, due to time out. Thus the victim will reassemble the fragments and will receive the attack whereas the IDS will not
make any noise or generate alerts.
In fact, the IDS (Intrussion Detection System) on the supernode tecnology design is not necessary because the first topology
introduce by the way is closely if the IDS attacks or compromised system methods are based first for the protocol OSI levels
(usually ,5,6 and 7) we don’t need take precaution for the simple rule of CyN kernel and protocol changes first, second by the
relationships for the “clustering” schema on that usually having fingerprints not standard.
The simple change on the fingerprints methods and protocol is the wall of some many exploitation methods…..Routing:
Incoming packets are dispatched to the correct protocol handler. For TCP and UDP, the configured services receive new
data and send responses if necessary. All outgoing packets are modified by the personality engine to mimic the behavior
of the configured network stack. The routing component is optional and used only when Honeyd simulates network topologies.8. Portability And Stability
CyN Operating system used for this project it was think for best performance on the supernode (nodes and guest systems) on
this hardware requirements:Two separated platforms
AMD x/64 Based chpset
512 -1 GB RAM Physical Memory
20 GB for each node and 60 GB for Supernode charter.
CD-ROM initial drive for instalation.Intel x/2 Xeon (2 processor sugested for supernode operator)
512 -1 GB RAM Physical Memory
20 GB for each node and 60 GB for Supernode charter.
CD-ROM initial drive for instalation.For guest machines:
Intel or AMD based systems having 100-500 megabytes of hard disk space for a minimal and around 1.5GB for full install
client (node) sofware.
9. ConclusionWhen i decided take this project around 2005 i don’t have any idea for where the project will be placed or used, but at 2006
seeing for network projects on the universities in germany, EU and spain, finally i decide implement myself the technology
adding a portion for other developments and ideas and design the principal schema for network operation. First thinking in
clustering but it’s not to satisfy my initial idea: “make a like closely seemed for neural network design”.
My expertise area of study are the cryptographycal scene, so i was make some network models for working on it, but it
project is not the same thing, because that i decide make a low cost schema for supernode technology to bring to my
community the chance of work with high processing level in a box. For the students who need more level of processing
and interaction between the machines models and human interface problems.
Also, i decide work for made a new changes on the existent models of networking nodes, supernode is the climb of the
newest network models used actually like kazaa, limewire clients but without the processing mode, only
