1xsbuilder.osc2002(3) User Contributed Perl Documentation xsbuilder.osc2002(3)
2
6 O'Reilly OpenSource Convention 2002
7 Gerald Richter
9 ecos gmbh
11 http://www.ecos.de
13
15 It's purpose is to automaticly generate a Perl interface to C Code
16 Solves the problem, that the Perl interface is not always uptodate with
18 the C interface
19 Saves a lot of Copy&Paste work
21 Systematical changes have to be done only once
23 For example changes in the memory management of strings.
25 Is part of mod_perl 2.0 build system
27 Most code is developed by Doug MacEachern. Additionaly I have
29 abstracted the code from mod_perl so it's useable for any C code
31 added new features like callbacks and the ability to parse comments
32 Replaced C::Scan with a Parse::RecDescent to be platform and compiler
33 independend
34 Goal is to replace the current mod_perl XS generation code with
36 XSBuilder
37 Inline versus XSBuilder
39 Inline: embed C-Code into Perl
41 XSBuilder: Create interface for existing C-libraries/applicationen
42
44 Create Perl functions/methods for every C function
45 The function can be assigned to different packages, also automaticly by
47 inspecting the first parameter
48 Create a Perl class for every C structure
50 Every element of structure becomes a Perl method to get/set it's value.
52 The object can be either a scalar reference (used by mod_perl) or a
53 reference to a hash (use by Embperl), which allows to store extra data
54 by the Perl code into this hash.
55 Create glue code to handle callbacks
57 There several sorts of callback, not all are implemented right now
59 Create Perl constant subs
61 Coming soon...
63
65 Parse the C header files
66 Extract
68 Functions, their arguments and return types
70 Structures and it's members
71 Constants
72 Callbacks
73 and create four tables which contains the results
75 Create the XS code
77 Input is
79 The source tables
81 Mapfiles which contains the mapping from C to Perl
82 Addtional C and Perl code that can be used to customize the interface
83 Output is
85 The XS files (one form every generated class)
87 Makefile.PL for every class
88 pm files
89
91 Create your own ParseSource class and override methods...
92 package Apache::DAV::ParseSource;
94 use strict;
96 use vars qw{@ISA $VERSION} ;
97 use ExtUtils::XSBuilder::ParseSource ;
98 @ISA = ('ExtUtils::XSBuilder::ParseSource') ;
100 $VERSION = '0.01';
101 my $dav_dir = 'C:\perl\msrc\cvs\mod_dav' ;
103 my $ap_dir = 'c:\programme\apache group\apache' ;
104 # ============================================================================
106 sub find_includes {
107 my $self = shift;
108 return $self->{includes} if $self->{includes};
109 my @includes = ("$ap_dir/include/ap_alloc.h", "$dav_dir/mod_dav.h") ;
110 return $self->{includes} = $self -> sort_includes (\@includes) ;
111 }
112 # ============================================================================
114 sub package { 'Apache::DAV' }
115 # ============================================================================
117 sub preprocess {
118 my $self = shift ;
119 $_[0] =~ s/(?:API_EXPORT)(?:_NONSTD)?\s*\(\s*(.*?)\s*\)/$1/g ;
120 }
121 1;
123 ...run it
125 use FindBin ;
127 use lib ($FindBin::Bin) ;
128 require ParseSource ;
130 Apache::DAV::ParseSource -> run ;
132 ...and you get
134 C:\perl\msrc\davint>perl xsbuilder\source_scan.pl
136 Will use mod_dav in C:\perl\msrc\cvs\mod_dav
137 Will use Apache in c:\programme\apache group\apache
138 Initialize parser
139 scan c:\programme\apache group\apache/include/ap_alloc.h ...
140 constant: APACHE_ALLOC_H
141 func: ap_init_alloc
142 func: ap_cleanup_alloc
143 func: ap_make_sub_pool
144 func: ap_destroy_pool
145 constant: ap_pool_join
146 func: ap_pool_join
147 func: ap_find_pool
148 func: ap_pool_is_ancestor
149 func: ap_clear_pool
150 func: ap_cleanup_for_exec
151 func: ap_palloc
152 func: ap_pcalloc
153 func: ap_pstrdup
154 func: ap_pstrndup
155 func: ap_pstrcat
156 func: ap_pvsprintf
157 valuefield: ap_pool * : pool
158 valuefield: int : elt_size
159 valuefield: int : nelts
160 valuefield: int : nalloc
161 valuefield: char * : elts
162 struct: (type=array_header)
163 ...
164 The result is stored in four tables
166 xsbuilder/tables/Apache/DAV/FuntionTable.pm
168 Contains all function, it arguments and comments
169 xsbuilder/tables/Apache/DAV/ConstantTable.pm
171 Contains all constants
172 xsbuilder/tables/Apache/DAV/StructureTable.pm
174 Contains all structures, it's members and their comments
175 xsbuilder/tables/Apache/DAV/CallbackTable.pm
177 Contains all callback function definitions
178
180 Mapfiles are used to tell XSBuilder how C datatypes, structures and
181 function aruments should be mapped into Perl ones.
182 Create your own WrapXS class and override methods
184 package Apache::DAV::WrapXS ;
186 use strict;
187 use vars qw{@ISA $VERSION} ;
188 use ExtUtils::XSBuilder::WrapXS ;
190 @ISA = ('ExtUtils::XSBuilder::WrapXS') ;
192 $VERSION = '0.01';
193 # ============================================================================
195 sub new_parsesource { [ Apache::DAV::ParseSource->new ] }
196 # ============================================================================
198 sub my_xs_prefix { 'davxs_' }
199 # ============================================================================
201 sub h_filename_prefix { 'moddav_xs_' }
202 # ============================================================================
204 sub xs_includes {
205 my $self = shift ;
206 my $i = $self -> SUPER::xs_includes ;
207 my @i = grep (!/ap_alloc/, @$i) ;
208 return \@i ;
209 }
210 XSBuilder can create/update initial maps for you
212 use FindBin ;
214 use lib ($FindBin::Bin) ;
215 require ParseSource ;
217 require WrapXS ;
218 Apache::DAV::WrapXS->checkmaps (' ');
220 run it
222 C:\perl\msrc\davint>perl xsbuilder\xs_check.pl
224 Will use mod_dav in C:\perl\msrc\cvs\mod_dav
225 Will use Apache in c:\programme\apache group\apache
226 Parse xsbuilder\maps/_types.map...
227 WARNING: No *_function.map file found in xsbuilder\maps
228 WARNING: No *_callback.map file found in xsbuilder\maps
229 WARNING: No *_structure.map file found in xsbuilder\maps
230 Write xsbuilder\maps/new_function.map...
231 Write xsbuilder\maps/new_callback.map...
232 Write xsbuilder\maps/new_structure.map...
233 Write xsbuilder\maps/new_type.map...
234 Now we have four map files
236 new_types.map
238 Contains the mapping from C type to Perl classes
239 new_functions.map
241 Contains the mapping form C functions to Perl functions. Can be
242 used to reorder arguments, tell XSBuilder which arguments are actu‐
243 aly return values and in which Perl package the function will be
244 created.
245 new_structures.map
247 Contains the mapping from C structures to Perl classes and defines
248 for which members a access methods should be created. You can also
249 specify if you want a "new" method for the class.
250 new_callbacks.map
252 Contains the mapping form C callback functions to Perl callback
253 functions. Can be used to reorder arguments, tell XSBuilder which
254 arguments are actualy return values and in which Perl package the
255 function will be created.
256 It's a good idea to rename the prefix from "new_" to something unique,
258 here we use "dav"
259 Everytime you rerun checkmaps, XSBuilder will create new_* files with
261 the items that are not already part of the other maps.
262 Next step is to customize the maps...
264
266 autogenerated dav_type.map
267 DIR ⎪
269 FILE ⎪
270 HANDLE ⎪
271 array_header ⎪
272 dav_buffer ⎪
273 dav_dyn_context ⎪
274 dav_dyn_hooks ⎪
275 dav_dyn_module ⎪
276 dav_dyn_provider ⎪
277 dav_error ⎪
278 dav_get_props_result ⎪
279 dav_hooks_liveprop ⎪
280 dav_hooks_locks ⎪
281 dav_hooks_repository ⎪
282 dav_if_header ⎪
283 dav_if_state_type ⎪
284 ...
285 Add Perl classes
287 struct array_header ⎪ Apache::Array
289 struct dav_buffer ⎪
290 struct dav_datum ⎪ Apache::DAV::Datum
291 struct dav_dyn_context ⎪ Apache::DAV::DynContext
292 struct dav_dyn_hooks ⎪ Apache::DAV::DynHooks
293 struct dav_dyn_module ⎪ Apache::DAV::DynModule
294 struct dav_dyn_provider ⎪ Apache::DAV::DynProvider
295 struct dav_error ⎪ Apache::DAV::Error
296 struct dav_get_props_result ⎪ Apache::DAV::PropsResult
297 struct dav_hooks_db ⎪ Apache::DAV::HooksDb
298 struct dav_hooks_liveprop ⎪ Apache::DAV::HooksLiveprop
299 struct dav_hooks_locks ⎪ Apache::DAV::HooksLocks
300 struct dav_hooks_repository ⎪ Apache::DAV::HooksRepository
301 struct dav_hooks_vsn ⎪
302 struct dav_if_header ⎪ Apache::DAV::IfHeader
303 struct dav_if_state_list ⎪ Apache::DAV::StateList
304 ...
305 struct pool ⎪ Apache::Pool
306 struct request_rec ⎪ Apache::
307 struct server_rec ⎪ Apache::Server
308 ...
309 Defines the mapping from C datatypes to Perl datatypes and classes and
311 tells XSBuilder which datatype are (like) structures
312
314 Function map defines the mapping from C functions arguments to Perl
315 arguments
316 Tell XSBuilder where to place functions and which prefix to strip
318 MODULE=Apache::DAV PACKAGE=guess PREFIX=dav_
319 Simple entries in the function map will be mapped 1:1 from C to Perl
321 dav_add_response
322 dav_buffer_append
323 dav_buffer_init
324 dav_buffer_place
325 dav_buffer_place_mem
326 dav_check_bufsize
327 dav_close_propdb
328 dav_collect_liveprop_uris
329 dav_dyn_module_add
330 dav_empty_elem
331 ...
332 The following map file entry tells XSBuilder that the value of "res_p"
334 should be returned
335 dav_get_resource ⎪ ⎪ r, <res_p
336 The C function
338 int dav_get_resource(request_rec *r, dav_resource **res_p) ;
340 How to call it from Perl
342 # get a sub request of the actual document
344 $subr = $r -> lookup_uri($uri);
345 # get a mod_dav resource object
347 my ($err, $resource) = $subr->get_resource;
348 You can let XSBuilder insert your custom code, for the interface
350 If you call "get_props" from Perl "glue_dav_get_props" will be
351 called, which can adjust the arguments and return types as neces‐
352 sary. The actual code for "glue_dav_get_props" will be taken from
353 separate include file.
354 dav_get_props ⎪ glue_
356 dav_get_allprops ⎪ glue_
357 xsinclude\Apache\DAV\PropResult\Apache__DAV__PropResults.h
359 dav_get_props_result * dav_glue_get_props(dav_propdb * db, dav_xml_doc *doc)
361 {
362 dav_get_props_result * result = (dav_get_props_result *)ap_palloc (db -> p, sizeof (dav_get_props_result)) ;
363 *result = dav_get_props(db, doc) ;
364 return result ;
365 }
366 Arguments can be replaced
368 MODULE=Apache::Array PACKAGE=Apache::Array PREFIX=ap_
369 ap_make_array ⎪ ap_make_array(r->pool, nelts, elt_size) ⎪ request_rec *:r, nelts, elt_size
371 ap_make_array requires a pool a it's first parameter, we pass the
373 request_rec from Perl and XSBuilder will take the pool from the
374 request_rec.
375
377 MALLOC=strdup:$dest = ($type)strdup($src)
378 FREE=strdup:free($src)
379 <dav_xml_elem>
381 name
382 ns
383 lang
384 first_cdata
385 following_cdata
386 parent
387 next
388 first_child
389 attr
390 last_child
391 ns_scope
392 propid
393 provider
394 ns_map
395 new
396 </dav_xml_elem>
397 MALLOC=strdup:$dest = ($type)ap_pstrdup(obj -> pool, $src)
399 <array_header>
401 pool
402 elt_size
403 nelts
404 nalloc
405 elts
406 private
407 </array_header>
408 Create a accessor functions for every element and, if requested, a new
410 method
411 $setprop = Apache::DAV::XMLElem -> new ({name => 'prop'}) ;
413 $elem = Apache::DAV::XMLElem -> new ({name => $name, ns => $namespaces}) ;
414 $setprop -> first_child($elem) ;
415 $first = $setprop -> first_child ;
416 some datatypes, like strings, requires dynamic allocated memory
418 From _types.map,which conatins a set of standard types
420 int ⎪ IV
422 int * ⎪ UNDEFINED
423 unsigned int ⎪ UV
424 signed int ⎪ IV
425 long ⎪ IV
426 long int ⎪ IV
427 unsigned long ⎪ UV
428 unsigned ⎪ UV
429 char * ⎪ PV ⎪ ⎪ ⎪ strdup
431 const char * ⎪ PV ⎪ ⎪ ⎪ strdup
432 const char ** ⎪ UNDEFINED
433 char const * ⎪ PV ⎪ ⎪ ⎪ strdup
434 unsigned char * ⎪ PV ⎪ ⎪ ⎪ strdup
435 const unsigned char * ⎪ PV ⎪ ⎪ ⎪ strdup
436 ...
437
439 Callback maps have the same options a function maps
440 # dav_hooks_db -> open
442 dav_error *(*)(pool * p,const dav_resource * resource,int ro,dav_db * * pdb) ⎪ p, resource, ro=0, <pdb
444 # dav_hooks_lock -> has_locks
446 dav_error *(*)(dav_lockdb * lockdb,const dav_resource * resource,int * locks_present)
448
450 use FindBin ;
451 use lib ($FindBin::Bin) ;
452 require ParseSource ;
454 require WrapXS ;
455 Apache::DAV::WrapXS->run;
457 ...and run...
459 C:\perl\msrc\davint>perl xsbuilder\xs_generate.pl
461 Will use mod_dav in C:\perl\msrc\cvs\mod_dav
462 Will use Apache in c:\programme\apache group\apache
463 Parse xsbuilder\maps/_types.map...
464 Parse xsbuilder\maps/dav_type.map...
465 mkdir xs
466 writing...xs//typemap
467 Parse xsbuilder\maps/dav_functions.map...
468 WARNING: Cannot map type int(*)(void * ,const char * ,const char * ) for function ap_table_do
469 WARNING: Cannot map type dav_buffer * for function dav_buffer_append
470 WARNING: Cannot map type dav_buffer * for function dav_buffer_init
471 WARNING: Cannot map type dav_buffer * for function dav_buffer_place
472 WARNING: Cannot map type dav_buffer * for function dav_buffer_place_mem
473 WARNING: Cannot map type dav_buffer * for function dav_check_bufsize
474 WARNING: Cannot map return type int * for function dav_collect_liveprop_uris
475 WARNING: Cannot map type dav_resource * * for function dav_ensure_resource_writable
476 WARNING: Cannot map type dav_buffer * for function dav_lock_get_activelock
477 WARNING: Cannot map type dav_buffer * for function dav_set_bufsize
478 WARNING: Cannot map type int * for function dav_xml2text
479 struct array_header...
480 Parse xsbuilder\maps/dav_structure.map...
481 elt_size...
482 nelts...
483 nalloc...
484 elts...
485 struct dav_buffer...
486 struct dav_datum...
487 dptr...
488 dsize...
489 struct dav_dyn_context...
490
492 We need create a top level Makefile.PL
493 use ExtUtils::MakeMaker ();
495 my $apdir = '/path/to/apache';
497 my $davdir = '/path/to/moddav';
498 %MMARGS = (
500 'INC' => "-I\"$davdir\" -I\"$apdir/include\" -I\"$apdir/os/unix\" -I\"$dir/xs\" -I\"$dir/xsinclude\"",
501 ) ;
502 open FH, ">xs/mmargs.pl" or die "Cannot open xs/mmargs.pl ($!)" ;
504 print FH Data::Dumper -> Dump ([\%MMARGS], ['MMARGS']) ;
505 close FH ;
506 ExtUtils::MakeMaker::WriteMakefile(
508 'NAME' => 'Apache::DAV',
509 'VERSION' => '0.13',
510 %MMARGS,
511 );
512 Makefile.PL's for all class are generated automaticly
514 C:\perl\msrc\davint>perl Makefile.PL
516 Will use Apache in c:\programme\apache group\apache
517 Will use mod_dav in C:\perl\msrc\cvs\mod_dav
518 Checking if your kit is complete...
519 Looks good
520 Writing Makefile for Apache::Array
521 Writing Makefile for Apache::DAV::Datum
522 Writing Makefile for Apache::DAV::DynContext
523 Writing Makefile for Apache::DAV::DynHooks
524 Writing Makefile for Apache::DAV::DynModule
525 Writing Makefile for Apache::DAV::DynProvider
526 Writing Makefile for Apache::DAV::Error
527 Writing Makefile for Apache::DAV::HooksDb
528 Writing Makefile for Apache::DAV::HooksLiveprop
529 Writing Makefile for Apache::DAV::HooksLocks
530 Writing Makefile for Apache::DAV::HooksRepository
531 Writing Makefile for Apache::DAV::IfHeader
532 Writing Makefile for Apache::DAV::Lock
533 Writing Makefile for Apache::DAV::LockDB
534 Writing Makefile for Apache::DAV::LockTokenList
535 Writing Makefile for Apache::DAV::LockupResult
536 Writing Makefile for Apache::DAV::PropCtx
537 Writing Makefile for Apache::DAV::PropsResult
538 Writing Makefile for Apache::DAV::Resource
539 Writing Makefile for Apache::DAV::Response
540 Writing Makefile for Apache::DAV::StateList
541 Writing Makefile for Apache::DAV::Text
542 Writing Makefile for Apache::DAV::TextHeader
543 Writing Makefile for Apache::DAV::WalkerCtx
544 Writing Makefile for Apache::DAV::XMLAttr
545 Writing Makefile for Apache::DAV::XMLDoc
546 Writing Makefile for Apache::DAV::XMLElem
547 Writing Makefile for Apache::DAV
548 Writing Makefile for Apache::TableEntry
549 Writing Makefile for Apache
550 Writing Makefile for WrapXS
551 Writing Makefile for Apache::DAV
552 and now compile...
554
556 Generating documentation
557 XSBuilder already extracts source comments for functions and struc‐
559 tures. It also parses doxygen comments, which are used in Apache 2.0.
560 Lyle Brooks has started on automaticly createing POD files from this
561 information.
562 Improving callbacks
564 Callbacks are the main area that needs improvement.
566 Bring it back to mod_perl 2.0
568 First version will be released just after the conference to CPAN
570 Any feedback and help appreciated
572 Questions?
574perl v5.8.8 2005-08-31 xsbuilder.osc2002(3)