Transcript Erlang

PCInt
A framework for servers and clients
PCInt
• P(ost) C(orrelator) Int(egrator)
• Highspeed correlator readout
– 32 correlatorboards in parallel
– to different storage nodes
• UniBoard correlator will need this too
• Multithreaded C++ code
• Focus on distributed client-server
architecture
• Focus on simple protocols with a finite
amount of messages/protocol
Basic stuff
• channels
– for moving information between its endpoints
– open(), read(), write(), close(), local(), remote()
– TCP/IP, UDP/IP, UNIX, file, stdio
• channel servers (ChanServ)
– listen for connection requests in separate thread
– create channel upon succesfull accept and keep it in a list
– support for TCP/IP, UDP/IP, UNIX
• RXTXBlock
– binary block of bytes for transmitting/receiving over channels
– threadsafe reference counted storage [mem. gets automatically deleted]
• Transportables
– information that moves over channels
– implement (de)serialization: toBlock(), fromBlock() [RXTXBlock]
More advanced objects
• ChannelHandlers
– baseclass for service servers
– deal with the traffic of a single channel
– handleIncoming() virtual method automatically called for
each block of data received on the channel
• ChannelManager
– monitor a number of ChanServs for newly accepted
channels since last check
– spawn one ChannelHandler-thread per incoming connection
• ServiceClient
– baseclass for ...
– thin wrapper around a channel, provides asynchronous
send/receive in separate thread
• sounded like a great idea at the time ... but it isn’t
– derived class’ methods become RPCs [mostly]
Defining a new service
class SLServer: public ChannelHandler {
public:
SLServer( /* constructor arguments if you want them */ );
virtual ChannelHandler* clone( void ) const;
virtual void
acceptChannel( void );
virtual void
handleIncoming(RXTXBlock& b,
channel& c);
virtual ~SLServer() ;
private:
/* datamembers you need to offer the service */
};
// Using your new service
SLServer
myserver( /*arguments */ );
std::vector<string> listenaddrs( “socket/tcp:4000”, “socket/unix:/tmp/slserver”, ... );
ChannelManager mgr(listenaddrs, myserver);
// done!
Under the hood
DoublerServer::handleIncoming(RXTXBlock b, channel& c) {
TInt
num2double;
TDouble rv;
num2double.fromBlock(b); // de-serialize number
rv = 2*num2double.get();
// prepare return value
c << rv.toBlock();
}
double DoublerServiceClient::doubleNumber( int num ) {
TInt
tnum( num );
TDouble reply;
this->sendBlock( tnum.toBlock() ); // serialize + send number
reply.fromBlock( myRcvBuf.front() ); // de-serialize received block
return reply.get();
}
// “.get()” returns contained type [typesafe!]
struct mystruct {
int
ival;
string sval;
} ;
#include <CTransportable.h>
typedef TMystruct CTransportable<mystruct>;
// CTransportable::myObject ’s type is “ref.counted pointer to template parameter”
template<> RXTXBlock CTransportable<mystruct>::toBlock( void ) {
TInt
ti( myObject->ival );
TString ts( myObject->sval );
return ti.toBlock()+ts.toBlock ();
}
// de-serialization returns true if succesfully de-serialized
template <> bool CTransportable<mystruct>::fromBlock(const RXTXBlock& b ) {
TInt
ti;
TString ts;
if( ti.fromBlock(b) && ts.fromBlock(b(ti.size()))) {
myObject->ival = ti.get();
myObject->sval = ts.get();
return true;
}
return false;
}
Note:
only defined the serialization of a single object!
Now we get for free:
// Transportable container
#include <TContainer.h>
// Like to send a list of mystructs?
typedef TLMystruct TContainer<mystruct, CTransportable, std::list>;
void sendList( const std::list<mystruct>& lms, channel& c ) {
TLMystruct
tlms( lms );
c << tlms.toBlock();
}
// Or would rather send a vector?
typedef TVMystruct TContainer<mystruct, CTransportable, std::vector>;
TContainer uses template template parameters, it takes templates as parameters:
template <typename T,
template <typename> class serializer,
template <typename> class container>
class TContainer { .... };
Messages
Message
m; // note: “Message is-a Transportable” [derived from]
RXTXBlock b;
TInt
ti;
TString
ts;
// the following works because of the reference-counted storage (shallow copy, yet MT safe)
m.addField( ti );
m.addField( ts );
ti = 42;
ts = “Hello world”;
channel << m.toBlock(); // that’s it: “42” and “Hello world” are sent.
// receiving:
channel >> b;
m.fromBlock( b ); // “ti” and “ts”’s content get overwritten with the deserialized contents of “b”
cout << ti.get() << “, “ << ts.get();
class MyMessage: public Message {
public:
MyMessage(int ival, const std::list<mystruct>& lms);
private:
IMessageID msgId; // integer message identifier
TInt
theInt;
TLMystruct wossName;
};
MyMessage::MyMessage(int ival, const std::list<mystruct>& lms ):
msgId( 0xABADDEED ), theInt( ival ), wossName( lms )
{
this->addField(msgId);
this->addField(theInt);
this->addField(wossName);
}
The MessageID classes are initialized with a value.
- upon sending this value is sent
- upon receiving, this value is matched: fromBlock()==true iff that value was received
bool SLServer::handleIncoming( const RXTXBlock& received, channel& c) {
// These messages all have (distinctive) MessageID fields ....
SLQuery
qrymsg;
SLQuerySysname
qrysysname;
SLRegisterService regmsg;
SLDeRegisterService deregmsg;
// therefore the following works nicely:
if( regmsg.fromBlock(received) ) {
// Register a new service-server
} else if( deregmsg.fromBlock(received) ) {
// De-register a service-server
} else if( qrymsg.fromBlock(received) ) {
// Query the service-locator for a certain servicetype
SLQueryResult
reply;
SLQuery::querytype_t qryType( qrymsg.queryType() );
reply = this->doQuery( qryType, match );
c << reply.toBlock();
} else if( ... ) {
...
}
return true;
}
Reusability
- Features a system-independant ServiceLocator
- unaware of what services it maintains + regex in queries
- Remote startup of processes
- allows to start/kill processes + status enquiry
- Transportable library very powerfull
- also allow for simple textbased protocols (Telnet)
- System DESIGNED to handle distributed readout
- little if no attention at all to format of data it needs to store
- Channels, Handlers, Utilities, Transportables easily reused
- only internal dependencies (e.g. Transportable -> channel)
- RL reUsecases of PCInt code
- evlbi (jive5a), wwpad
- LCS did copy/import elements
Portability
Verified to interoperate on ILP32 and LP64 platforms
ILP32: Linux/i386, Solaris/SPARC
LP64:
Linux/amd64
Will break on systems where int = 64bits
unlikely we will run on those systems
Should work on any reasonable POSIX compliant system
Mac OSX 32+64bit seem to work - spent 2 hours
However ...
Code could use a brush-up
- coding style possibly considered awkward, fixable
Learnt a lot of new things over the last few years
- parts could do with re-implementation/simplification
- very strict compartmenting of code helps
- import handy stuff from jive5a (eVLBI code)
Remove MT where it doesn’t add anything
- most notably in the ServiceClient baseclass
- and in the remote-startup daemon
More convenience functions should be added