|
 |
|
WP5: Technology Development
Overview of WP5
Workpackage 5 of EuroGrid was designed to build on the funcionality of
UNICORE and to extend it to provide the middleware necessary for the
other workpackages.
The spread of applications in Workpackages 1 to 4
is such that the middleware developed in Workpackage 5 is
applicable to many applications in a computational Grid.
Five areas of technology have been identified where support was
improved in order to make EUROGRID suitable for use in the
domain-specific GRIDs (WP 1 to WP4), and to improve the prospects of
EUROGRID take-up and exploitation in science and industry. They are
described below as tasks T5.1 to T5.5.
There was clearly overlap with other European GRID projects and
informal contacts were developed to provide a robust solution for European Grids.
T5.1: Efficient data transfer
Since EuroGrid technology is designed to be used over Internet routes,
the issue of fast and secure data transfer is of great
importance. Several considerations arise.
- Optimisation of transfer bandwidth and cost. There may be more
than one route for a point-to-point file transfer. There may be
trade-offs between guarentees of bandwidth and cost. For example, ATM
PVC routes provide guarenteed bandwidth and very low latency but are
chargable. A decision may be made between an Internet route offering
variable bandwidth or a dedicated link.
- Fail-safe and encrypted transfer. If a network link goes down in
a transfer, the files should not be corrupted without warning. It is
also possible to investigate switching the transfer to alternative
routes. Security of data over open networks is an important concern,
especially for commercial use of the Grid.
- Overlap of transfer and processing. We investigate this in terms
of real-time processing. This form of file transfer needs to be
"application aware". An example of a series of experiments
investigating the use of such techniques over intercontinental networks
can be found in the paper by Pickles et al.
which was chosen the Best Paper of
HPCN2000.
In general, application requirements on data transfer fall in two
categories: burst transfers, where latency is critical and bulk
transfer where the available bandwidth should be utilised. Both forms
of transfer are important in a Computational Grid and both will be
investigated in this workpackage.
T5.2: Resource broker
A resource broker in a computatational grid, where there are a variety
of machines with specialist architectures, needs to be able to access
and make decisions based on both static and dynamic information. By
static information we mean resources such as the type of
architecture, range of compilers and software, data storage
facilities, that remain constant over relatively long periods of
time. By dynamic information we mean the loading of machines, versions
of compilers and software, current scheduling and charging
strategies. These can change rapidly, in some cases continuously
(e.g. loading of the machine).
The resource broker needs this information to match a user's
computational requirements, traditionally represented as a "job", to
what is available on the GRID. Also the broker needs to be able to
negotiate to balance the clients needs for low cost and fast
turnaround, for example.
The broker has to read and update information about the clients
allowances and accounts on the machines across the GRID. For example,
if the brokering process results in a job being submitted to Site A
and Resources A, B, C are used in the job (say A is CPU time, B is
disk space use, and C is archival space required for the results),
then the broker needs to inform the client of cost and allow the site
to have some means of recouping the cost. This could be via an account
kept at the site, say a client has X amount of resource tokens
representing Y euros worth of resource, then the account needs to be
decremented by the amount used. Or the site might not hold a permanent
account for the user but would bill him/her for the job.
Thus a resource broker for a computatational Grid is a sophisticated
software concept. This workpackage will be developed on the basis of a
resource economy which has been running successfully at the CSAR WWW
service for two years. The knowledge and account updating tools of
this service provide a starting point for developing the full
brokering functionality. More details are at
CSAR WWW site
T5.3 ASP Infrastructure
The cost of a job submitted on the GRID may include the cost of the
software licence necessary for the work. This particularly applies to
commercial usage where the cost of the licence for powerful software
may dominate the cost for the job.
It may be that an organisation needs to use software on large machines
but could not justify purchasing a machine and a software licence for
its exclusive use. There is an opportunity for Application Service
Provider (ASP) to make available such software on a pay-per-use basis.
It can clearly be seen how such a model fits with the concept of the
resource broker in task T5.2. The combination of middleware provided
through T5.2 and T5.3 provides a basis for commercial and industrial
use of a computational GRID, either as a company-wide Intranet or via
independent ASPs over the Internet.
T5.4 Application coupling
Many large scale computational applications involve interacting
sub-problems which may be best located on different machines. A good
class of examples are systems where fluids interact with solid
structures and differenent solvers are optimal for the fluid dynamics
and structural mechanics. Another example are environmental
simualtions which may involve solving equations for gaseous, fluid and
solid behaviour along with chemical and perhaps biological reactions.
In computational GRIDS with a range of specialist architectures,
different submodels may be placed on machines where the solvers run
particularly efficiently, this is know as "machine affinity".
Task 5.4 will investigate integrating a communication middleware
for such coupled applications. There needs to be some "weak"
sychronicity for such sub-model coupling. Thus we will investigat
interfaces to schedulers for co-scheduling of machines and other
resources (archives and staging areas for file storage and transfer
for example).
T5.5 Interactive access
Some computational work requires interactive access to resources. An
example would be the running of a simulation with visualization of the
results being provided as the simulation runs. This can be extended to
situations where the simulation is steered according to the visualized
output.
This approach is becoming increasingly important to a range of
industrial, scientific and civil users of computational
facilities. Rapid virtual prototyping of new products needs design and
engineering teams to be able to observe the behaviour of the proposed
product and visual information is very important to this
process. Civil emergency procedures, training simulations and
environmental impact studies also rely on the interaction of skilled
personnel with a simulation based on observed or hypothetical data.
Most large scale computational facilities have traditionally operated
their machines in batch mode and the current model of UNICORE is based
on this approach. Workpackage T5.5 will add functionality to allow for
the possibility of interactive use of computational and visualization
facilities.
d.mallmann@
fz-juelich.de
10-May-2004
URL: <http://www.eurogrid.org/wp5.html>
|
|
