1xsbuilder.osc2002(3) User Contributed Perl Documentation xsbuilder.osc2002(3)
2
6 O'Reilly OpenSource Convention 2002
7 Gerald Richter
9 ecos gmbh
11 http://www.ecos.de
13
15 It's purpose is to automaticly generate a Perl interface to C Code
16 Solves the problem, that the Perl interface is not always uptodate with the
17 C interface
18 Saves a lot of Copy&Paste work
19 Systematical changes have to be done only once
20 For example changes in the memory management of strings.
21 Is part of mod_perl 2.0 build system
23 Most code is developed by Doug MacEachern. Additionaly I have
24 abstracted the code from mod_perl so it's useable for any C code
26 added new features like callbacks and the ability to parse comments
27 Replaced C::Scan with a Parse::RecDescent to be platform and compiler
28 independend
29 Goal is to replace the current mod_perl XS generation code with
31 XSBuilder
32 Inline versus XSBuilder
34 Inline: embed C-Code into Perl
35 XSBuilder: Create interface for existing C-libraries/applicationen
36
38 Create Perl functions/methods for every C function
39 The function can be assigned to different packages, also automaticly by
40 inspecting the first parameter
41 Create a Perl class for every C structure
43 Every element of structure becomes a Perl method to get/set it's value.
44 The object can be either a scalar reference (used by mod_perl) or a
45 reference to a hash (use by Embperl), which allows to store extra data
46 by the Perl code into this hash.
47 Create glue code to handle callbacks
49 There several sorts of callback, not all are implemented right now
50 Create Perl constant subs
52 Coming soon...
53
55 Parse the C header files
56 Extract
57 Functions, their arguments and return types
59 Structures and it's members
60 Constants
61 Callbacks
62 and create four tables which contains the results
64 Create the XS code
66 Input is
67 The source tables
69 Mapfiles which contains the mapping from C to Perl
70 Addtional C and Perl code that can be used to customize the interface
71 Output is
73 The XS files (one form every generated class)
75 Makefile.PL for every class
76 pm files
77
79 Create your own ParseSource class and override methods...
80 package Apache::DAV::ParseSource;
81 use strict;
83 use vars qw{@ISA $VERSION} ;
84 use ExtUtils::XSBuilder::ParseSource ;
85 @ISA = ('ExtUtils::XSBuilder::ParseSource') ;
87 $VERSION = '0.01';
88 my $dav_dir = 'C:\perl\msrc\cvs\mod_dav' ;
90 my $ap_dir = 'c:\programme\apache group\apache' ;
91 # ============================================================================
93 sub find_includes {
94 my $self = shift;
95 return $self->{includes} if $self->{includes};
96 my @includes = ("$ap_dir/include/ap_alloc.h", "$dav_dir/mod_dav.h") ;
97 return $self->{includes} = $self -> sort_includes (\@includes) ;
98 }
99 # ============================================================================
101 sub package { 'Apache::DAV' }
102 # ============================================================================
104 sub preprocess {
105 my $self = shift ;
106 $_[0] =~ s/(?:API_EXPORT)(?:_NONSTD)?\s*\(\s*(.*?)\s*\)/$1/g ;
107 }
108 1;
111 ...run it
113 use FindBin ;
114 use lib ($FindBin::Bin) ;
115 require ParseSource ;
117 Apache::DAV::ParseSource -> run ;
119 ...and you get
121 C:\perl\msrc\davint>perl xsbuilder\source_scan.pl
122 Will use mod_dav in C:\perl\msrc\cvs\mod_dav
123 Will use Apache in c:\programme\apache group\apache
124 Initialize parser
125 scan c:\programme\apache group\apache/include/ap_alloc.h ...
126 constant: APACHE_ALLOC_H
127 func: ap_init_alloc
128 func: ap_cleanup_alloc
129 func: ap_make_sub_pool
130 func: ap_destroy_pool
131 constant: ap_pool_join
132 func: ap_pool_join
133 func: ap_find_pool
134 func: ap_pool_is_ancestor
135 func: ap_clear_pool
136 func: ap_cleanup_for_exec
137 func: ap_palloc
138 func: ap_pcalloc
139 func: ap_pstrdup
140 func: ap_pstrndup
141 func: ap_pstrcat
142 func: ap_pvsprintf
143 valuefield: ap_pool * : pool
144 valuefield: int : elt_size
145 valuefield: int : nelts
146 valuefield: int : nalloc
147 valuefield: char * : elts
148 struct: (type=array_header)
149 ...
150 The result is stored in four tables
152 xsbuilder/tables/Apache/DAV/FuntionTable.pm
153 Contains all function, it arguments and comments
154 xsbuilder/tables/Apache/DAV/ConstantTable.pm
156 Contains all constants
157 xsbuilder/tables/Apache/DAV/StructureTable.pm
159 Contains all structures, it's members and their comments
160 xsbuilder/tables/Apache/DAV/CallbackTable.pm
162 Contains all callback function definitions
163
165 Mapfiles are used to tell XSBuilder how C datatypes, structures and
166 function aruments should be mapped into Perl ones.
167 Create your own WrapXS class and override methods
168 package Apache::DAV::WrapXS ;
169 use strict;
170 use vars qw{@ISA $VERSION} ;
171 use ExtUtils::XSBuilder::WrapXS ;
173 @ISA = ('ExtUtils::XSBuilder::WrapXS') ;
175 $VERSION = '0.01';
176 # ============================================================================
178 sub new_parsesource { [ Apache::DAV::ParseSource->new ] }
179 # ============================================================================
181 sub my_xs_prefix { 'davxs_' }
182 # ============================================================================
184 sub h_filename_prefix { 'moddav_xs_' }
185 # ============================================================================
187 sub xs_includes {
188 my $self = shift ;
189 my $i = $self -> SUPER::xs_includes ;
190 my @i = grep (!/ap_alloc/, @$i) ;
191 return \@i ;
192 }
193 XSBuilder can create/update initial maps for you
195 use FindBin ;
196 use lib ($FindBin::Bin) ;
197 require ParseSource ;
199 require WrapXS ;
200 Apache::DAV::WrapXS->checkmaps (' ');
202 run it
204 C:\perl\msrc\davint>perl xsbuilder\xs_check.pl
205 Will use mod_dav in C:\perl\msrc\cvs\mod_dav
206 Will use Apache in c:\programme\apache group\apache
207 Parse xsbuilder\maps/_types.map...
208 WARNING: No *_function.map file found in xsbuilder\maps
209 WARNING: No *_callback.map file found in xsbuilder\maps
210 WARNING: No *_structure.map file found in xsbuilder\maps
211 Write xsbuilder\maps/new_function.map...
212 Write xsbuilder\maps/new_callback.map...
213 Write xsbuilder\maps/new_structure.map...
214 Write xsbuilder\maps/new_type.map...
215 Now we have four map files
217 new_types.map
218 Contains the mapping from C type to Perl classes
219 new_functions.map
221 Contains the mapping form C functions to Perl functions. Can be
222 used to reorder arguments, tell XSBuilder which arguments are
223 actualy return values and in which Perl package the function will
224 be created.
225 new_structures.map
227 Contains the mapping from C structures to Perl classes and defines
228 for which members a access methods should be created. You can also
229 specify if you want a "new" method for the class.
230 new_callbacks.map
232 Contains the mapping form C callback functions to Perl callback
233 functions. Can be used to reorder arguments, tell XSBuilder which
234 arguments are actualy return values and in which Perl package the
235 function will be created.
236 It's a good idea to rename the prefix from "new_" to something unique,
238 here we use "dav"
239 Everytime you rerun checkmaps, XSBuilder will create new_* files with
241 the items that are not already part of the other maps.
242 Next step is to customize the maps...
245 autogenerated dav_type.map
246 DIR |
247 FILE |
248 HANDLE |
249 array_header |
250 dav_buffer |
251 dav_dyn_context |
252 dav_dyn_hooks |
253 dav_dyn_module |
254 dav_dyn_provider |
255 dav_error |
256 dav_get_props_result |
257 dav_hooks_liveprop |
258 dav_hooks_locks |
259 dav_hooks_repository |
260 dav_if_header |
261 dav_if_state_type |
262 ...
263 Add Perl classes
265 struct array_header | Apache::Array
266 struct dav_buffer |
267 struct dav_datum | Apache::DAV::Datum
268 struct dav_dyn_context | Apache::DAV::DynContext
269 struct dav_dyn_hooks | Apache::DAV::DynHooks
270 struct dav_dyn_module | Apache::DAV::DynModule
271 struct dav_dyn_provider | Apache::DAV::DynProvider
272 struct dav_error | Apache::DAV::Error
273 struct dav_get_props_result | Apache::DAV::PropsResult
274 struct dav_hooks_db | Apache::DAV::HooksDb
275 struct dav_hooks_liveprop | Apache::DAV::HooksLiveprop
276 struct dav_hooks_locks | Apache::DAV::HooksLocks
277 struct dav_hooks_repository | Apache::DAV::HooksRepository
278 struct dav_hooks_vsn |
279 struct dav_if_header | Apache::DAV::IfHeader
280 struct dav_if_state_list | Apache::DAV::StateList
281 ...
282 struct pool | Apache::Pool
283 struct request_rec | Apache::
284 struct server_rec | Apache::Server
285 ...
286 Defines the mapping from C datatypes to Perl datatypes and classes and
288 tells XSBuilder which datatype are (like) structures
289
291 Function map defines the mapping from C functions arguments to Perl
292 arguments
293 Tell XSBuilder where to place functions and which prefix to strip
295 MODULE=Apache::DAV PACKAGE=guess PREFIX=dav_
296 Simple entries in the function map will be mapped 1:1 from C to Perl
298 dav_add_response
299 dav_buffer_append
300 dav_buffer_init
301 dav_buffer_place
302 dav_buffer_place_mem
303 dav_check_bufsize
304 dav_close_propdb
305 dav_collect_liveprop_uris
306 dav_dyn_module_add
307 dav_empty_elem
308 ...
309 The following map file entry tells XSBuilder that the value of "res_p"
311 should be returned
312 dav_get_resource | | r, <res_p
313 The C function
315 int dav_get_resource(request_rec *r, dav_resource **res_p) ;
317 How to call it from Perl
319 # get a sub request of the actual document
321 $subr = $r -> lookup_uri($uri);
322 # get a mod_dav resource object
324 my ($err, $resource) = $subr->get_resource;
325 You can let XSBuilder insert your custom code, for the interface
327 If you call "get_props" from Perl "glue_dav_get_props" will be
328 called, which can adjust the arguments and return types as
329 necessary. The actual code for "glue_dav_get_props" will be taken
330 from separate include file.
331 dav_get_props | glue_
333 dav_get_allprops | glue_
334 xsinclude\Apache\DAV\PropResult\Apache__DAV__PropResults.h
336 dav_get_props_result * dav_glue_get_props(dav_propdb * db, dav_xml_doc *doc)
338 {
339 dav_get_props_result * result = (dav_get_props_result *)ap_palloc (db -> p, sizeof (dav_get_props_result)) ;
340 *result = dav_get_props(db, doc) ;
341 return result ;
342 }
343 Arguments can be replaced
345 MODULE=Apache::Array PACKAGE=Apache::Array PREFIX=ap_
346 ap_make_array | ap_make_array(r->pool, nelts, elt_size) | request_rec *:r, nelts, elt_size
348 ap_make_array requires a pool a it's first parameter, we pass the
350 request_rec from Perl and XSBuilder will take the pool from the
351 request_rec.
352
354 MALLOC=strdup:$dest = ($type)strdup($src)
355 FREE=strdup:free($src)
356 <dav_xml_elem>
358 name
359 ns
360 lang
361 first_cdata
362 following_cdata
363 parent
364 next
365 first_child
366 attr
367 last_child
368 ns_scope
369 propid
370 provider
371 ns_map
372 new
373 </dav_xml_elem>
374 MALLOC=strdup:$dest = ($type)ap_pstrdup(obj -> pool, $src)
376 <array_header>
378 pool
379 elt_size
380 nelts
381 nalloc
382 elts
383 private
384 </array_header>
385 Create a accessor functions for every element and, if requested, a new
387 method
388 $setprop = Apache::DAV::XMLElem -> new ({name => 'prop'}) ;
389 $elem = Apache::DAV::XMLElem -> new ({name => $name, ns => $namespaces}) ;
390 $setprop -> first_child($elem) ;
391 $first = $setprop -> first_child ;
392 some datatypes, like strings, requires dynamic allocated memory
394 From _types.map,which conatins a set of standard types
395 int | IV
397 int * | UNDEFINED
398 unsigned int | UV
399 signed int | IV
400 long | IV
401 long int | IV
402 unsigned long | UV
403 unsigned | UV
404 char * | PV | | | strdup
406 const char * | PV | | | strdup
407 const char ** | UNDEFINED
408 char const * | PV | | | strdup
409 unsigned char * | PV | | | strdup
410 const unsigned char * | PV | | | strdup
411 ...
412
414 Callback maps have the same options a function maps
415 # dav_hooks_db -> open
417 dav_error *(*)(pool * p,const dav_resource * resource,int ro,dav_db * * pdb) | p, resource, ro=0, <pdb
419 # dav_hooks_lock -> has_locks
422 dav_error *(*)(dav_lockdb * lockdb,const dav_resource * resource,int * locks_present)
424
426 use FindBin ;
427 use lib ($FindBin::Bin) ;
428 require ParseSource ;
430 require WrapXS ;
431 Apache::DAV::WrapXS->run;
433 ...and run...
435 C:\perl\msrc\davint>perl xsbuilder\xs_generate.pl
436 Will use mod_dav in C:\perl\msrc\cvs\mod_dav
437 Will use Apache in c:\programme\apache group\apache
438 Parse xsbuilder\maps/_types.map...
439 Parse xsbuilder\maps/dav_type.map...
440 mkdir xs
441 writing...xs//typemap
442 Parse xsbuilder\maps/dav_functions.map...
443 WARNING: Cannot map type int(*)(void * ,const char * ,const char * ) for function ap_table_do
444 WARNING: Cannot map type dav_buffer * for function dav_buffer_append
445 WARNING: Cannot map type dav_buffer * for function dav_buffer_init
446 WARNING: Cannot map type dav_buffer * for function dav_buffer_place
447 WARNING: Cannot map type dav_buffer * for function dav_buffer_place_mem
448 WARNING: Cannot map type dav_buffer * for function dav_check_bufsize
449 WARNING: Cannot map return type int * for function dav_collect_liveprop_uris
450 WARNING: Cannot map type dav_resource * * for function dav_ensure_resource_writable
451 WARNING: Cannot map type dav_buffer * for function dav_lock_get_activelock
452 WARNING: Cannot map type dav_buffer * for function dav_set_bufsize
453 WARNING: Cannot map type int * for function dav_xml2text
454 struct array_header...
455 Parse xsbuilder\maps/dav_structure.map...
456 elt_size...
457 nelts...
458 nalloc...
459 elts...
460 struct dav_buffer...
461 struct dav_datum...
462 dptr...
463 dsize...
464 struct dav_dyn_context...
465
467 We need create a top level Makefile.PL
468 use ExtUtils::MakeMaker ();
469 my $apdir = '/path/to/apache';
471 my $davdir = '/path/to/moddav';
472 %MMARGS = (
474 'INC' => "-I\"$davdir\" -I\"$apdir/include\" -I\"$apdir/os/unix\" -I\"$dir/xs\" -I\"$dir/xsinclude\"",
475 ) ;
476 open FH, ">xs/mmargs.pl" or die "Cannot open xs/mmargs.pl ($!)" ;
479 print FH Data::Dumper -> Dump ([\%MMARGS], ['MMARGS']) ;
480 close FH ;
481 ExtUtils::MakeMaker::WriteMakefile(
484 'NAME' => 'Apache::DAV',
485 'VERSION' => '0.13',
486 %MMARGS,
487 );
488 Makefile.PL's for all class are generated automaticly
490 C:\perl\msrc\davint>perl Makefile.PL
491 Will use Apache in c:\programme\apache group\apache
492 Will use mod_dav in C:\perl\msrc\cvs\mod_dav
493 Checking if your kit is complete...
494 Looks good
495 Writing Makefile for Apache::Array
496 Writing Makefile for Apache::DAV::Datum
497 Writing Makefile for Apache::DAV::DynContext
498 Writing Makefile for Apache::DAV::DynHooks
499 Writing Makefile for Apache::DAV::DynModule
500 Writing Makefile for Apache::DAV::DynProvider
501 Writing Makefile for Apache::DAV::Error
502 Writing Makefile for Apache::DAV::HooksDb
503 Writing Makefile for Apache::DAV::HooksLiveprop
504 Writing Makefile for Apache::DAV::HooksLocks
505 Writing Makefile for Apache::DAV::HooksRepository
506 Writing Makefile for Apache::DAV::IfHeader
507 Writing Makefile for Apache::DAV::Lock
508 Writing Makefile for Apache::DAV::LockDB
509 Writing Makefile for Apache::DAV::LockTokenList
510 Writing Makefile for Apache::DAV::LockupResult
511 Writing Makefile for Apache::DAV::PropCtx
512 Writing Makefile for Apache::DAV::PropsResult
513 Writing Makefile for Apache::DAV::Resource
514 Writing Makefile for Apache::DAV::Response
515 Writing Makefile for Apache::DAV::StateList
516 Writing Makefile for Apache::DAV::Text
517 Writing Makefile for Apache::DAV::TextHeader
518 Writing Makefile for Apache::DAV::WalkerCtx
519 Writing Makefile for Apache::DAV::XMLAttr
520 Writing Makefile for Apache::DAV::XMLDoc
521 Writing Makefile for Apache::DAV::XMLElem
522 Writing Makefile for Apache::DAV
523 Writing Makefile for Apache::TableEntry
524 Writing Makefile for Apache
525 Writing Makefile for WrapXS
526 Writing Makefile for Apache::DAV
527 and now compile...
530 Generating documentation
531 XSBuilder already extracts source comments for functions and
532 structures. It also parses doxygen comments, which are used in Apache
533 2.0. Lyle Brooks has started on automaticly createing POD files from
534 this information.
535 Improving callbacks
537 Callbacks are the main area that needs improvement.
538 Bring it back to mod_perl 2.0
540 First version will be released just after the conference to CPAN
541 Any feedback and help appreciated
542 Questions?
543perl v5.28.0 2002-08-05 xsbuilder.osc2002(3)