working on redis db reorganization.

ChangeLog

@@ -1,7 +1,8 @@
 10/26/2014 Oleg Moskalenko <mom040267@gmail.com>
-Version 4.2.2.1 'Monza':
+Version 4.2.2.2 'Monza':
 	- Black- and white- IP lists are divided per realm
 		(the DB schema for those two tables changed);
+	- Updated Redis database schema.
 	- TCP/TLS tests extended.
 	- Relay RTCP sockets ports allocation fixed.
 	- List of libraries cleaned.

INSTALL

@@ -844,7 +844,7 @@ Of course, the administrators can play with the connection string as they want.
 
 When starting the turnserver, the psql-userdb parameter will be, for example:
 
-turnserver ... --psql-userdb="host=localhost dbname=turn user=turn password=turn connect_timeout=30"
+turnserver ... --psql-userdb="host=localhost dbname=coturn user=turn password=turn connect_timeout=30"
 
 Or, for 9.x PostgreSQL versions: 
 turnserver ... --psql-userdb=postgresql://username:password@/databasename ...
@@ -858,20 +858,20 @@ Fill in users, for example:
 
   Shared secret for the TURN REST API:
   
-  $ bin/turnadmin -s logen -e "host=localhost dbname=turn user=turn password=turn"
+  $ bin/turnadmin -s logen -e "host=localhost dbname=coturn user=turn password=turn"
   
   Long-term credentials mechanism:
   
-  $ bin/turnadmin -a -e "host=localhost dbname=turn user=turn password=turn" -u gorst -r north.gov -p hero
+  $ bin/turnadmin -a -e "host=localhost dbname=coturn user=turn password=turn" -u gorst -r north.gov -p hero
-  $ bin/turnadmin -a -e "host=localhost dbname=turn user=turn password=turn" -u ninefingers -r north.gov -p youhavetoberealistic
+  $ bin/turnadmin -a -e "host=localhost dbname=coturn user=turn password=turn" -u ninefingers -r north.gov -p youhavetoberealistic
   
   Long-term credentials mechanism with SHA256 extention:
-  $ bin/turnadmin -a -e "host=localhost dbname=turn user=turn password=turn" -u bethod -r north.gov -p king-of-north --sha256
+  $ bin/turnadmin -a -e "host=localhost dbname=coturn user=turn password=turn" -u bethod -r north.gov -p king-of-north --sha256
   
   Short-term credentials mechanism:
    
-  $ bin/turnadmin -A -e "host=localhost dbname=turn user=turn password=turn" -u gorst -r north.gov -p hero
+  $ bin/turnadmin -A -e "host=localhost dbname=coturn user=turn password=turn" -u gorst -r north.gov -p hero
-  $ bin/turnadmin -A -e "host=localhost dbname=turn user=turn password=turn" -u ninefingers -r north.gov -p youhavetoberealistic 
+  $ bin/turnadmin -A -e "host=localhost dbname=coturn user=turn password=turn" -u ninefingers -r north.gov -p youhavetoberealistic 
 
 XV. MySQL (MariaDB) setup
 
@@ -913,20 +913,20 @@ the root account.
 
   Shared secret for the TURN REST API:
   
-  $ bin/turnadmin -s logen -M "host=localhost dbname=turn user=turn password=turn"
+  $ bin/turnadmin -s logen -M "host=localhost dbname=coturn user=turn password=turn"
   
   Long-term credentials mechanism:
   
-  $ bin/turnadmin -a -M "host=localhost dbname=turn user=turn password=turn" -u gorst -r north.gov -p hero
+  $ bin/turnadmin -a -M "host=localhost dbname=coturn user=turn password=turn" -u gorst -r north.gov -p hero
-  $ bin/turnadmin -a -M "host=localhost dbname=turn user=turn password=turn" -u ninefingers -r north.gov -p youhavetoberealistic
+  $ bin/turnadmin -a -M "host=localhost dbname=coturn user=turn password=turn" -u ninefingers -r north.gov -p youhavetoberealistic
   
   Long-term credentials mechanism with SHA256 extention:
-  $ bin/turnadmin -a -M "host=localhost dbname=turn user=turn password=turn" -u bethod -r north.gov -p king-of-north --sha256
+  $ bin/turnadmin -a -M "host=localhost dbname=coturn user=turn password=turn" -u bethod -r north.gov -p king-of-north --sha256
   
   Short-term credentials mechanism:
    
-  $ bin/turnadmin -A -M "host=localhost dbname=turn user=turn password=turn" -u gorst -r north.gov -p hero
+  $ bin/turnadmin -A -M "host=localhost dbname=coturn user=turn password=turn" -u gorst -r north.gov -p hero
-  $ bin/turnadmin -A -M "host=localhost dbname=turn user=turn password=turn" -u ninefingers -r north.gov -p youhavetoberealistic 
+  $ bin/turnadmin -A -M "host=localhost dbname=coturn user=turn password=turn" -u ninefingers -r north.gov -p youhavetoberealistic 
 
 7) Now we can use mysql in the turnserver.
 
@@ -944,11 +944,11 @@ The format of the MySQL connection string is:
 So, an example of the MySQL database parameter in the TURN server command 
 line would be:
 
---mysql-userdb="host=localhost dbname=turn user=turn password=turn connect_timeout=30"
+--mysql-userdb="host=localhost dbname=coturn user=turn password=turn connect_timeout=30"
 
 Or in the turnserver.conf file:
 
-mysql-userdb="host=localhost dbname=turn user=turn password=turn connect_timeout=30"
+mysql-userdb="host=localhost dbname=coturn user=turn password=turn connect_timeout=30"
 
 If you have to use a secure MySQL connection (SSL) then you have to use also
 the optional connection string parameters for the secure communications:
@@ -977,11 +977,11 @@ http://hergert.me/docs/mongo-c-driver/mongoc_uri.html:
 So, an example of the MongoDB database parameter in the TURN server command 
 line would be:
 
---mongo-userdb="mongodb://localhost:27017/turndb"
+--mongo-userdb="mongodb://localhost:27017/coturn"
 
 Or in the turnserver.conf file:
 
-mongo-userdb="mongodb://localhost:27017/turndb"
+mongo-userdb="mongodb://localhost:27017/coturn"
 
 The meanings of the MongoDB keys are the same as for the other databases, see the 
 explanations for the Postgres, for example.
@@ -1012,11 +1012,11 @@ connection string is:
 So, an example of the Redis database parameter in the TURN server command 
 line would be:
 
---redis-userdb="ip=127.0.0.1 dbname=0 password=turn connect_timeout=30"
+--redis-userdb="ip=127.0.0.1 dbname=2 password=turn connect_timeout=30"
 
 Or in the turnserver.conf file:
 
-redis-userdb="ip=127.0.0.1 dbname=0 password=turn connect_timeout=30"
+redis-userdb="ip=127.0.0.1 dbname=2 password=turn connect_timeout=30"
 
 Redis can be also used for the TURN allocation status check and for status and 
 traffic notifications.
@@ -1042,20 +1042,20 @@ Redis TURN admin commands:
 
   Shared secret for the TURN REST API:
   
-  $ bin/turnadmin -s logen -N "host=localhost dbname=0 user=turn password=turn"
+  $ bin/turnadmin -s logen -N "host=localhost dbname=2 user=turn password=turn"
   
   Long-term credentials mechanism:
   
-  $ bin/turnadmin -a -N "host=localhost dbname=0 user=turn password=turn" -u gorst -r north.gov -p hero
+  $ bin/turnadmin -a -N "host=localhost dbname=2 user=turn password=turn" -u gorst -r north.gov -p hero
-  $ bin/turnadmin -a -N "host=localhost dbname=0 user=turn password=turn" -u ninefingers -r north.gov -p youhavetoberealistic
+  $ bin/turnadmin -a -N "host=localhost dbname=2 user=turn password=turn" -u ninefingers -r north.gov -p youhavetoberealistic
   
   Long-term credentials mechanism with SHA256 extention:
-  $ bin/turnadmin -a -N "host=localhost dbname=0 user=turn password=turn" -u bethod -r north.gov -p king-of-north --sha256
+  $ bin/turnadmin -a -N "host=localhost dbname=2 user=turn password=turn" -u bethod -r north.gov -p king-of-north --sha256
   
   Short-term credentials mechanism:
    
-  $ bin/turnadmin -A -N "host=localhost dbname=0 user=turn password=turn" -u gorst -r north.gov -p hero
+  $ bin/turnadmin -A -N "host=localhost dbname=2 user=turn password=turn" -u gorst -r north.gov -p hero
-  $ bin/turnadmin -A -N "host=localhost dbname=0 user=turn password=turn" -u ninefingers -r north.gov -p youhavetoberealistic 
+  $ bin/turnadmin -A -N "host=localhost dbname=2 user=turn password=turn" -u ninefingers -r north.gov -p youhavetoberealistic 
   
 See the file testredisdbsetup.sh for the data structure examples.
 

rpm/build.settings.sh

@@ -2,7 +2,7 @@
 
 # Common settings script.
 
-TURNVERSION=4.2.2.1
+TURNVERSION=4.2.2.2
 BUILDDIR=~/rpmbuild
 ARCH=`uname -p`
 TURNSERVER_SVN_URL=http://coturn.googlecode.com/svn

rpm/turnserver.spec

@@ -1,5 +1,5 @@
 Name:		turnserver
-Version:	4.2.2.1
+Version:	4.2.2.2
 Release:	0%{dist}
 Summary:	Coturn TURN Server
 
@@ -295,7 +295,7 @@ fi
 
 %changelog
 * Sun Oct 26 2014 Oleg Moskalenko <mom040267@gmail.com>
-  - Sync to 4.2.2.1
+  - Sync to 4.2.2.2
 * Sun Oct 05 2014 Oleg Moskalenko <mom040267@gmail.com>
   - Sync to 4.2.1.2
 * Thu Aug 14 2014 Oleg Moskalenko <mom040267@gmail.com>

src/apps/relay/dbdrivers/dbd_redis.c

@@ -396,19 +396,14 @@ static int set_redis_realm_opt(char *realm, const char* key, unsigned long *valu
 
 ///////////////////////////////////////////////////////////////////////////////////////////////////////////
 
-static int redis_get_auth_secrets(secrets_list_t *sl, u08bits *realm) {
+static int redis_get_auth_secrets(secrets_list_t *sl, u08bits *realm)
+{
 	int ret = -1;
 	redisContext *rc = get_redis_connection();
 	if (rc) {
-		redisReply *reply = (redisReply*)redisCommand(rc, "keys turn/realm/%s/secret/*", (char*)realm);
+		redisReply *reply = (redisReply*) redisCommand(rc, "smembers turn/realm/%s/secret", (char*) realm);
 		if (reply) {
 
-			secrets_list_t keys;
-			size_t isz = 0;
-			char s[257];
-
-			init_secrets_list(&keys);
-
 			if (reply->type == REDIS_REPLY_ERROR)
 				TURN_LOG_FUNC(TURN_LOG_LEVEL_ERROR, "Error: %s\n", reply->str);
 			else if (reply->type != REDIS_REPLY_ARRAY) {
@@ -417,28 +412,10 @@ static int redis_get_auth_secrets(secrets_list_t *sl, u08bits *realm) {
 			} else {
 				size_t i;
 				for (i = 0; i < reply->elements; ++i) {
-					add_to_secrets_list(&keys,reply->element[i]->str);
+					add_to_secrets_list(sl, reply->element[i]->str);
 				}
 			}
 
-			for(isz=0;isz<keys.sz;++isz) {
-				snprintf(s,sizeof(s),"get %s", keys.secrets[isz]);
-				redisReply *rget = (redisReply *)redisCommand(rc, s);
-				if(rget) {
-					if (rget->type == REDIS_REPLY_ERROR)
-						TURN_LOG_FUNC(TURN_LOG_LEVEL_ERROR, "Error: %s\n", rget->str);
-					else if (rget->type != REDIS_REPLY_STRING) {
-						if (rget->type != REDIS_REPLY_NIL)
-							TURN_LOG_FUNC(TURN_LOG_LEVEL_ERROR, "Unexpected type: %d\n", rget->type);
-					} else {
-						add_to_secrets_list(sl,rget->str);
-					}
-					turnFreeRedisReply(rget);
-				}
-			}
-
-			clean_secrets_list(&keys);
-
 			ret = 0;
 
 			turnFreeRedisReply(reply);
@@ -780,16 +757,18 @@ static int redis_list_oauth_keys(void) {
   return ret;
 }
   
-static int redis_show_secret(u08bits *realm) {
+
+static int redis_show_secret(u08bits *realm)
+{
 	int ret = -1;
 	donot_print_connection_success = 1;
 	redisContext *rc = get_redis_connection();
 	if (rc) {
 		redisReply *reply = NULL;
 		if (realm && realm[0]) {
-			reply = (redisReply*)redisCommand(rc, "keys turn/realm/%s/secret/*",(char*)realm);
+			reply = (redisReply*) redisCommand(rc, "keys turn/realm/%s/secret", (char*) realm);
 		} else {
-			reply = (redisReply*)redisCommand(rc, "keys turn/realm/*/secret/*");
+			reply = (redisReply*) redisCommand(rc, "keys turn/realm/*/secret");
 		}
 		if (reply) {
 			secrets_list_t keys;
@@ -811,16 +790,21 @@ static int redis_show_secret(u08bits *realm) {
 			}
 
 			for (isz = 0; isz < keys.sz; ++isz) {
-				snprintf(s,sizeof(s),"get %s", keys.secrets[isz]);
+				snprintf(s, sizeof(s), "smembers %s", keys.secrets[isz]);
 				redisReply *rget = (redisReply *) redisCommand(rc, s);
 				if (rget) {
-					if (rget->type == REDIS_REPLY_ERROR)
+					if (rget->type == REDIS_REPLY_ERROR) {
 						TURN_LOG_FUNC(TURN_LOG_LEVEL_ERROR, "Error: %s\n", rget->str);
-					else if (rget->type != REDIS_REPLY_STRING) {
+					} else if (rget->type == REDIS_REPLY_STRING) {
+						printf("%s\n", rget->str);
+					} else if (rget->type != REDIS_REPLY_ARRAY) {
 						if (rget->type != REDIS_REPLY_NIL)
 							TURN_LOG_FUNC(TURN_LOG_LEVEL_ERROR, "Unexpected type: %d\n", rget->type);
 					} else {
-						printf("%s\n",rget->str);
+						size_t i;
+						for (i = 0; i < rget->elements; ++i) {
+							printf("%s\n", rget->element[i]->str);
+						}
 					}
 				}
 				turnFreeRedisReply(rget);
@@ -835,67 +819,23 @@ static int redis_show_secret(u08bits *realm) {
 	return ret;
 }
   
-static int redis_del_secret(u08bits *secret, u08bits *realm) {
+
+static int redis_del_secret(u08bits *secret, u08bits *realm)
+{
 	int ret = -1;
 	donot_print_connection_success = 1;
 	redisContext *rc = get_redis_connection();
 	if (rc) {
-		redisReply *reply = (redisReply*)redisCommand(rc, "keys turn/realm/%s/secret/*", (char*)realm);
+		turnFreeRedisReply(redisCommand(rc, "srem turn/realm/%s/secret %s", (char*) realm, (char*) secret));
-		if(reply) {
-			secrets_list_t keys;
-			size_t isz = 0;
-			char s[TURN_LONG_STRING_SIZE];
-
-			init_secrets_list(&keys);
-
-			if (reply->type == REDIS_REPLY_ERROR)
-				TURN_LOG_FUNC(TURN_LOG_LEVEL_ERROR, "Error: %s\n", reply->str);
-			else if (reply->type != REDIS_REPLY_ARRAY) {
-				if (reply->type != REDIS_REPLY_NIL)
-					TURN_LOG_FUNC(TURN_LOG_LEVEL_ERROR, "Unexpected type: %d\n", reply->type);
-			} else {
-				size_t i;
-				for (i = 0; i < reply->elements; ++i) {
-					add_to_secrets_list(&keys,reply->element[i]->str);
-				}
-			}
-
-			for(isz=0;isz<keys.sz;++isz) {
-				if(!secret || (secret[0]==0)) {
-					snprintf(s,sizeof(s),"del %s", keys.secrets[isz]);
-					turnFreeRedisReply(redisCommand(rc, s));
-				} else {
-					snprintf(s,sizeof(s),"get %s", keys.secrets[isz]);
-					redisReply *rget = (redisReply *)redisCommand(rc, s);
-					if(rget) {
-						if (rget->type == REDIS_REPLY_ERROR)
-							TURN_LOG_FUNC(TURN_LOG_LEVEL_ERROR, "Error: %s\n", rget->str);
-						else if (rget->type != REDIS_REPLY_STRING) {
-							if (rget->type != REDIS_REPLY_NIL)
-								TURN_LOG_FUNC(TURN_LOG_LEVEL_ERROR, "Unexpected type: %d\n", rget->type);
-						} else {
-							if(!strcmp((char*)secret,rget->str)) {
-								snprintf(s,sizeof(s),"del %s", keys.secrets[isz]);
-								turnFreeRedisReply(redisCommand(rc, s));
-							}
-						}
-						turnFreeRedisReply(rget);
-					}
-				}
-			}
-
 		turnFreeRedisReply(redisCommand(rc, "save"));
-
-			clean_secrets_list(&keys);
-
-			turnFreeRedisReply(reply);
 		ret = 0;
 	}
-	}
 	return ret;
 }
   
-static int redis_set_secret(u08bits *secret, u08bits *realm) {
+
+static int redis_set_secret(u08bits *secret, u08bits *realm)
+{
 	int ret = -1;
 	donot_print_connection_success = 1;
 	redisContext *rc = get_redis_connection();
@@ -904,7 +844,7 @@ static int redis_set_secret(u08bits *secret, u08bits *realm) {
 
 		redis_del_secret(secret, realm);
 
-		snprintf(s,sizeof(s),"set turn/realm/%s/secret/%lu %s", (char*)realm, (unsigned long)turn_time(), secret);
+		snprintf(s, sizeof(s), "sadd turn/realm/%s/secret %s", (char*) realm, secret);
 
 		turnFreeRedisReply(redisCommand(rc, s));
 		turnFreeRedisReply(redisCommand(rc, "save"));

src/ns_turn_defs.h

@@ -31,7 +31,7 @@
 #ifndef __IOADEFS__
 #define __IOADEFS__
 
-#define TURN_SERVER_VERSION "4.2.2.1"
+#define TURN_SERVER_VERSION "4.2.2.2"
 #define TURN_SERVER_VERSION_NAME "Monza"
 #define TURN_SOFTWARE "Coturn-" TURN_SERVER_VERSION " '" TURN_SERVER_VERSION_NAME "'"
 

turndb/schema.userdb.redis

@@ -18,14 +18,13 @@ are not bound to a realm). So, there will be key "turn/user/gorst/password"
 and the value will be "hero".
 
 3) For the shared secrets (REST API), several key/value pairs 
-may be used (same as in SQL schema). The key will be 
+may be used (same as in SQL schema). The secrets are stored as members 
-"turn/realm/<realm-name>/secret/<arbitrary secret ID>" and the value will be 
+of an unordered set. The name of the set will be 
-"<secret>". For example, if we have secrets "hero1", "hero2" and "hero3", 
+"turn/realm/<realm-name>/secret" and the value(s) will be the secret(s).
-then we will have keys "turn/realm/north.gov/secret/123",
+For example, if we have secrets "hero1", "hero2" and "hero3", 
-"turn/realm/north.gov/secret/234", "turn/realm/north.gov/secret/345" 
+then we will have set "turn/realm/north.gov/secret" with values
-and their values will be  "hero1", "hero2", "hero3". The turnserver will 
+"hero1", "hero2" and "hero3". The turnserver will try to use the 
-issue command "keys turn/realm/north.gov/secret/*" it it will try to use the 
+secrets in arbitrary order.
-obtained keys in arbitrary order.
 
 4) The "white" and "black" peer IP ranges are stored as keys of the
 following form: 
@@ -97,7 +96,7 @@ This example sets user database for:
   	with default realm "north.gov";
   * long-term credentials with open passwords and 
   	with default realm "north.gov";
-  * TURN REST API with shared secret "logen";
+  * TURN REST API with shared secrets "logen", etc;
   * short-term credentials mechanism, with open passwords;
   * Black and white IP peer lists used.
   * Information how to match ORIGIN field with extra
@@ -126,8 +125,8 @@ set turn/realm/north.gov/user/gorst/password "hero"
 set turn/realm/crinna.org/user/whirrun/password "sword"
 set turn/realm/crinna.org/user/stranger-come-knocking/password "civilization"
 
-set turn/realm/north.gov/secret/1368426581 "logen"
+sadd turn/realm/north.gov/secret "logen" "bloody9"
-set turn/realm/crinna.org/secret/777888999 "north"
+sadd turn/realm/crinna.org/secret "north" "library"
 
 set turn/user/ninefingers/password "youhavetoberealistic"
 set turn/user/gorst/password "hero"

turndb/testredisdbsetup.sh

@@ -21,8 +21,8 @@ set turn/realm/north.gov/user/bethod/password "king-of-north"
 set turn/realm/crinna.org/user/whirrun/password "sword"
 set turn/realm/crinna.org/user/stranger-come-knocking/password "civilization"
 
-set turn/realm/north.gov/secret/1368426581 "logen"
+sadd turn/realm/north.gov/secret "logen" "bloody9"
-set turn/realm/crinna.org/secret/777888999 "north"
+sadd turn/realm/crinna.org/secret "north" "library"
 
 set turn/user/ninefingers/password "youhavetoberealistic"
 set turn/user/gorst/password "hero"